system

Abstract

Provide a system. 【Solution means】 Means for receiving and analyzing requests in natural language from users, Means for collecting and analyzing the user's past purchase history and preference data based on the received requests, Means for generating optimal product proposals by utilizing a generative artificial intelligence, Means for customizing the proposed content using multimodal data, Means for presenting the generated product proposals to the user, receiving feedback, and responding, Means for setting up a natural language processing function and analyzing the input from the user, Means for obtaining the user's preference information by utilizing a data storage structure, Means for providing customized product proposals by utilizing a machine learning model, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional e-commerce system, it is common for a user to select a product based on their limited subjectivity and knowledge, which may result in the problem that they cannot find the products they truly need. In particular, as user preferences diversify, it is difficult for standard product proposals to meet individual needs, and there is a possibility that the purchasing desire may decline. Also, since the range of information provided by the user is limited, there is also the problem that it is difficult to make appropriate product proposals.

Means for Solving the Problems

[0005] To solve the above problems, the electronic commerce system of the present invention is equipped with means for receiving and analyzing requests from users in natural language, and for collecting and analyzing the user's purchase history and preference data. Furthermore, it is equipped with means for generating and presenting optimal product suggestions that meet the user's requests by utilizing generative artificial intelligence. Moreover, by utilizing multimodal data, it can identify visual preferences from additional information provided by the user, such as images and videos, and customize the suggested content. In addition, by further refining the suggestions based on user feedback, it realizes suggestions that truly meet the user's needs.

[0006] "User" refers to an individual or legal entity that uses an e-commerce system to search for and purchase goods or services.

[0007] "Natural language" refers to the forms of words and sentences that humans use on a daily basis, and in computer systems, it is treated as text data.

[0008] "Analysis" is the process of understanding the content of received data or information and revealing its structure.

[0009] "Past purchase history" refers to a record of products and services that a user has purchased in the past.

[0010] "Preference data" refers to information about a user's preferences, including data collected based on past behavior and preferences.

[0011] "Generative artificial intelligence" refers to artificial intelligence technology that has the ability to create new patterns and information based on large amounts of data.

[0012] "Optimal product recommendations" refer to presenting products and services that best match the user's needs and preferences and stimulate their desire to purchase.

[0013] "Multimodal data" refers to data that includes multiple formats such as text, images, and videos, and can be analyzed in an integrated manner.

[0014] "Visual preference" refers to a user's preference for visual elements such as color and design.

[0015] "Feedback" refers to user reactions and opinions, and is information used to adjust the system's operation and suggestions based on them.

[0016] "Customizing the proposal" refers to adjusting or modifying product or service suggestions according to the individual user's needs and preferences. [Brief explanation of the drawing]

[0017] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] The system of this invention is designed to be integrated into an e-commerce platform and to provide personalized product recommendations to users. A specific example of its operation is shown below.

[0039] First, the user accesses an e-commerce site through their device and sends an inquiry in natural language about the product or service they are considering purchasing. This operation is typically performed via a chat box or voice input interface.

[0040] When the server receives a request from a user, it first passes it to a natural language processing engine. Here, the user's inquiry is analyzed to clarify the intent and content of the request. For example, if a user enters "I want running shoes," the server extracts the keywords "running" and "shoes" and focuses on the sports shoes category.

[0041] Next, the server accesses the database to collect the user's past purchase history and preference data. This allows it to identify what products the user has previously purchased, as well as their preferred brands and styles. Based on this information, the generative artificial intelligence selects and suggests products that it believes are best suited to the user.

[0042] Furthermore, users can upload additional information using their devices, such as images and videos related to their purchases. The server analyzes this multimodal data and customizes product recommendations based on the user's visual preferences.

[0043] After product suggestions are generated, the server presents them to the user's terminal. The user has the opportunity to view the suggested product list and provide feedback. For example, if the user requests that "the design should be more colorful," the server uses the generating AI to re-evaluate the suggestions and present new, more appropriate products.

[0044] In this way, the system aims to provide product suggestions that more accurately meet user needs and improve the purchasing experience. Through this series of processes, the interaction between users and the e-commerce platform becomes more efficient and satisfying.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] Users access e-commerce platforms using their devices and enter requests for desired products or services in natural language. This is typically done through the site's chat window.

[0048] Step 2:

[0049] The server receives a request from the user. This information is sent to a natural language processing engine, which analyzes the request and clarifies its content and intent.

[0050] Step 3:

[0051] Based on the analyzed information, the server retrieves the user's past purchase history and preference data from a database. This enables product recommendations that take into account the user's preferences and past behavior.

[0052] Step 4:

[0053] The server uses artificial intelligence to integrate user requests with historical data to generate an optimal product list. This list includes recommended products and similar items.

[0054] Step 5:

[0055] Users upload images and videos related to products using their devices. This multimodal data is sent to the server as additional preference information.

[0056] Step 6:

[0057] The server analyzes multimodal data to identify the user's visual preferences. Based on this information, it customizes the suggested product list to create more specific product recommendations.

[0058] Step 7:

[0059] The server then displays the final product suggestions generated by the server to the user's terminal. The user can browse these products and provide further requests or feedback.

[0060] Step 8:

[0061] The server receives user feedback, uses AI to re-evaluate the suggested products based on that feedback, and then presents a further refined product list to provide suggestions that best meet the user's needs.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] Modern e-commerce platforms offer generic and uniform product suggestions to users, failing to adequately reflect each user's individual preferences and past purchase history. Therefore, there is a need to develop systems that can satisfy users' purchasing desires and assist them in making optimal product choices.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes means for receiving and analyzing requests from users in natural language, means for collecting and analyzing the user's past purchase history and preference information based on the received requests, and means for generating optimal product suggestions using generative artificial intelligence. This enables personalized product suggestions for each user and improves the purchasing experience.

[0067] "Requests made in natural language" refer to sentences or phrases that users use to communicate their intentions in text or voice, without being constrained by any specific format.

[0068] "Purchase history" refers to detailed information and records of products that a user has purchased in the past.

[0069] "Preference information" refers to data that shows a user's preferences and tendencies, and represents personal judgment criteria that influence product selection.

[0070] "Generative artificial intelligence" refers to an algorithm or system that has the ability to generate product suggestions tailored to the user using machine learning techniques.

[0071] "Diverse media data" refers to data in different formats, including visual information such as images and videos, and auditory information such as audio.

[0072] "Product recommendations" refer to a list of products and services recommended based on the user's needs and preferences.

[0073] "Opinions" refer to the thoughts, requests, and wishes that users express regarding the proposed product.

[0074] The embodiments for carrying out the present invention will now be described. This system implements advanced processing for providing personalized product recommendations to users. Its specific operation is shown below.

[0075] First, users use their devices to access e-commerce platforms and then submit inquiries in natural language about desired products or services they are interested in. These natural language inquiries are typically made through chat boxes or voice input interfaces.

[0076] Next, the server receives the user's request and first analyzes it using a natural language processing (NLP) engine. The analysis clarifies the user's intent and request. Based on this information, if the user says "I want running shoes," the server extracts the keywords "running" and "shoes" and identifies the product category to suggest.

[0077] Next, the server accesses an internal database to collect data based on the user's past purchase history and preferences. This allows the server to identify products the user has previously purchased, as well as their preferred brands and styles. This data serves as crucial foundational information for suggesting the most suitable products to the user.

[0078] Subsequently, the server uses a generative AI model to generate optimal product recommendations for the user based on this analytical data. The generative AI model is based on machine learning algorithms and lists the most suitable products while taking into account the user's past preferences.

[0079] Furthermore, users can upload images and videos related to their purchases through their devices. The server analyzes this multimodal data and further customizes product recommendations based on the user's visual preferences. In this process, preferences for color and design are particularly important factors.

[0080] Finally, the server displays the generated product suggestions to the user's device. The user has the opportunity to view the presented product list and provide feedback. This feedback will be used to further improve the suggestions.

[0081] For example, if a user enters "I want a tent I can use for camping," the server will refer to the user's past purchase history of outdoor equipment based on the keywords "camping" and "tent," and suggest appropriate products. An example of a prompt to the generating AI model is, "Please suggest new products related to the camping equipment the user has purchased in the past."

[0082] This will allow the system to accurately meet user needs and improve the purchasing experience.

[0083] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0084] Step 1:

[0085] Users use their devices to access e-commerce platforms and submit inquiries in natural language about products and services they are considering purchasing. This input can be in text or voice format and is sent from the device to the server. For example, a user might ask, "I want running shoes."

[0086] Step 2:

[0087] The server receives requests from users in natural language and passes them to a natural language processing (NLP) engine for analysis. The input text is extracted through analysis into keywords such as "running" and "shoes." This output clarifies the user's intent and request.

[0088] Step 3:

[0089] Based on user requests, the server accesses a database within the system to collect the user's past purchase history and preference information. Database queries are sent as input, the data is processed, and the output includes data on the user's preferences and past purchases. Specifically, this includes information on previously purchased sports shoes and brands.

[0090] Step 4:

[0091] The server utilizes a generative AI model to generate optimal product suggestions based on the user's past data and received requests. The AI ​​model uses machine learning algorithms to analyze a large amount of user data and suggest new products. The input for this step is past preference data and product category information, and the output is a personalized list of products.

[0092] Step 5:

[0093] Users can upload images and videos related to their purchases using their devices. The server receives these and analyzes them as multimodal data. The visual data used as input is analyzed to identify the user's visual preferences, which are then reflected in the most suitable recommendations. For example, if a user prefers colorful designs, that information is taken into account in the product recommendations.

[0094] Step 6:

[0095] The server presents customized product suggestions to the user's terminal. It uses a generated product list as input and employs prompts to present relevant information to the user. As output, the user receives the product list and can review the details. The user can then provide feedback and return data to the server to further improve the accuracy of the suggestions.

[0096] (Application Example 1)

[0097] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0098] Providing products that quickly and appropriately meet the diverse needs of users in online shopping is challenging. In this situation, simply suggesting products based on a user's past purchase history and basic information is insufficient; more advanced analysis and personalization are required. Furthermore, it is essential to flexibly update suggestions using user feedback. The ability to accurately grasp user needs using natural language input and multimodal data is also crucial. Solving these challenges is essential.

[0099] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0100] In this invention, the server includes means for receiving and analyzing requests in natural language from users, means for collecting and analyzing the user's past purchase history and preference data, means for generating optimal product suggestions using generative artificial intelligence, and means for providing customized product suggestions using machine learning models. This enables rapid and highly accurate product suggestions to meet the diverse needs of users.

[0101] "Requests in natural language" refer to requests or questions that users input using everyday language.

[0102] "Means of analysis" refers to a system or program that has the function of breaking down received information and performing a process to extract its meaning and intent.

[0103] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0104] "Preference data" refers to information about preferences and trends collected based on a user's past preferences, interests, and usage history.

[0105] "Generative artificial intelligence" refers to machine learning techniques and algorithms that dynamically generate optimal suggestions and answers based on user requests and preferences.

[0106] "Multimodal data" refers to information that combines multiple types of data, such as audio, images, and text.

[0107] "Means of customization" refers to methods and technologies for adjusting and optimizing output according to the individual user's needs and preferences.

[0108] "Feedback" refers to the information provided by users regarding their user experience and suggestions, and is used to improve the system's performance.

[0109] "Natural language processing capabilities" refer to the technology that enables computers to understand and process human language.

[0110] A "data storage structure" refers to a system of databases and file systems designed to efficiently store and utilize information.

[0111] A "machine learning model" refers to an algorithm that learns from large amounts of data and recognizes patterns to make future predictions and classifications.

[0112] In order to implement this invention, it is mainly necessary for a server and a user terminal to work together. The server has advanced computing resources and is responsible for processing user requests and generating appropriate product suggestions. User terminals refer to various devices such as smartphones and computers, and are means for users to interface with the system.

[0113] When a server receives a request from a user in natural language, it first uses a natural language processing engine to analyze the request. Typically, this involves utilizing machine learning-based natural language processing technologies such as the OpenAI® API. Based on the analysis results, the server identifies the user's intent and emotions regarding the request.

[0114] Next, the server retrieves the user's past purchase history and preference data from the database. This data is managed by a database management system such as SQLite. Once the information is collected, the server uses machine learning models, such as TENSORFLOW® or PyTorch, to create generative AI models and make optimal product recommendations.

[0115] Furthermore, if multimodal data such as images and videos are available from the user, this data is used to further customize the suggestions. At this time, computer vision technology and deep learning are used to analyze the user's visual preferences and generate new feedback.

[0116] The generated product suggestions are sent to the user's device, where the user evaluates them. If feedback is submitted, for example, if the user suggests "a more colorful design would be better," the server re-evaluates the suggestion and updates it. In this way, the system can continuously provide the best suggestions that meet the user's needs.

[0117] As a concrete example, the following is an example of a prompt message that analyzes the request, "I want lightweight, waterproof running shoes."

[0118] Example of a prompt:

[0119] "Analyze the user's query: 'I want lightweight, waterproof running shoes,' and identify the requested product category and characteristics."

[0120] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0121] Step 1:

[0122] The user uses a device to input a request using natural language, either by voice or text. This request clearly indicates the user's purchasing interest and is specific, such as "I want lightweight, waterproof running shoes." The device then sends this request to the server.

[0123] Step 2:

[0124] The server receives requests written in natural language and analyzes them using a natural language processing engine. This analysis breaks down the user's request into keywords, extracting information such as "lightweight," "waterproof," and "running shoes." The input is the user's request, and the output is a set of analyzed keywords.

[0125] Step 3:

[0126] The server then retrieves and aggregates user purchase history and preference data from the database. At this stage, a database management system such as SQLite is used. The input is the user's ID, and the output is the user's past purchase history and preference data. This information is used to process the data and analyze user interests and preferences.

[0127] Step 4:

[0128] The server uses a machine learning model to generate optimal product suggestions based on the analysis results and user preference data. Here, a generative AI model is used to generate a list of products best suited to satisfy the user's requirements. The input is the output data from steps 2 and 3, and the output is a list of product suggestions.

[0129] Step 5:

[0130] If the user submits additional images or videos, this data is analyzed and multimodal data is used to further customize product suggestions. Image processing techniques are used to analyze the user's visual preferences. The input is image or video data from the user, and the output is a list of more customized product suggestions.

[0131] Step 6:

[0132] The generated product suggestions are sent to the user's terminal, where the user views the suggested products and provides feedback. The server receives the feedback, re-evaluates them, and generates suggestions again if necessary. The input is the user's feedback, and the output is the revised product suggestions.

[0133] Step 7:

[0134] The server performs a process based on this feedback, updating the generating AI model to improve the accuracy of future suggestions. The server constantly learns from new data to improve the performance of product suggestions.

[0135] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0136] The system of the present invention is designed to provide personalized product recommendations to users on an e-commerce platform, and aims to provide a more advanced user experience by incorporating emotion recognition technology.

[0137] Users first use their device to inquire about the products or services they wish to purchase using natural language via a chat box or voice input interface on the e-commerce platform.

[0138] The server receives inquiries from users and uses a natural language processing engine to analyze the content and intent of the requests. Simultaneously, an emotion engine recognizes the user's emotional state from their input, identifying emotions such as joy, surprise, and sadness. This allows the server to understand what emotions are behind the user's requests.

[0139] Based on the analyzed requests and emotional states, the server collects the user's past purchase history and preference data from a database, integrates and analyzes this information, and then applies emotional information to customize the recommendations.

[0140] Furthermore, it utilizes generative artificial intelligence to generate optimal product recommendations. Here, the emotional state identified by the emotion engine is used to determine the priority of product recommendations and to highlight specific product groups. For example, if the user expresses the emotion of "surprise," unique and new products will be prioritized in the recommendations.

[0141] The generated product suggestions are further customized by incorporating multimodal data. If a user uploads images or videos from their device, that data is analyzed, and the suggestions are adjusted to match the user's visual preferences.

[0142] Finally, the server presents the generated suggestions to the user's terminal. The user can view the suggested products and provide additional feedback. The server incorporates this feedback and adjusts the suggestions as needed to provide an experience that recommends the most appropriate products to meet the user's needs. For example, if a user requests "I want a bag for everyday use," and the emotion engine determines that the user is experiencing "enjoyment," then stylishly designed and colorful products will be prioritized in the suggestions.

[0143] The following describes the processing flow.

[0144] Step 1:

[0145] Users access e-commerce platforms using their devices, inputting the products or services they wish to purchase in natural language, and submitting inquiries. Chat windows and voice input are used.

[0146] Step 2:

[0147] The server receives a request written in natural language from the user. This request is then passed to a natural language processing engine for analysis.

[0148] Step 3:

[0149] The server uses a natural language processing engine to analyze the user's request, clarifying its intent and requirements. This analysis identifies key keywords and desired product categories.

[0150] Step 4:

[0151] The server uses an emotion engine to recognize the emotional state contained in the user's natural language request. Emotions such as joy, surprise, and sadness are identified, and the user's mental state is recorded.

[0152] Step 5:

[0153] The server retrieves the user's past purchase history and preference data from the database. This data reflects the user's preferences and past purchasing behavior and can be used to make recommendations.

[0154] Step 6:

[0155] The server integrates historical data collected with emotional information obtained from the emotion engine, and uses generative artificial intelligence to generate optimal product suggestions. Here, the suggested products are adjusted according to the user's emotional state.

[0156] Step 7:

[0157] Users can upload images and videos related to products from their devices. This multimodal data is sent to the server to complement the user's visual preferences.

[0158] Step 8:

[0159] The server analyzes multimodal data to identify the user's visual preferences and then customizes the suggestions accordingly. This makes the product list more closely match the user's specific desires.

[0160] Step 9:

[0161] The server then displays the final product suggestions to the user's terminal. The user can view the suggested products and provide further feedback.

[0162] Step 10:

[0163] The server receives user feedback, and based on that feedback, the AI ​​generates suggestions to re-evaluate and adjust them, continuously presenting the product that best suits the user's needs.

[0164] (Example 2)

[0165] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0166] In modern e-commerce platforms, personalized product recommendations are crucial. However, simply basing recommendations on past purchase history and preferences fails to adequately reflect users' actual needs and emotions, limiting their ability to improve satisfaction. Furthermore, when suggesting appropriate products based on user input, emotional states are often overlooked, making it difficult to provide an optimal purchasing experience.

[0167] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0168] In this invention, the server includes means for receiving natural language input from the user and analyzing its content, means for collecting and analyzing information on the user's purchase history and preferences based on the received input, and means for generating appropriate product suggestions using artificial intelligence. This makes it possible to provide more accurate and personalized product suggestions that also take into account the user's emotional information.

[0169] "User-generated natural language input" refers to users conveying information to an electronic platform using expressions from everyday conversation.

[0170] "Means for analyzing content" refers to a device or software that analyzes received natural language input and processes it to understand its meaning and intent.

[0171] "Purchase history and preference information" refers to data related to products and services that a user has previously acquired, as well as their preferences for those products and services.

[0172] "Generative artificial intelligence" refers to an intelligent program or algorithm that automatically generates optimal product and service suggestions based on large amounts of data.

[0173] "Data in diverse formats" refers to a collection of data that encompasses information obtained in different media formats, such as text, images, audio, and video.

[0174] "Visual preference" refers to a user's particular preferences regarding visual elements such as colors, designs, and styles.

[0175] "Emotional information" refers to information that indicates an emotional state, inferred from user input and interactions.

[0176] The present invention provides a system that generates personalized product suggestions when a user purchases goods or services on an e-commerce platform using a terminal, by having the user input requests in natural language and having the server analyze and process those inputs.

[0177] Users can make product requests via text or voice through their device. If a user enters a request such as "I want new sneakers," the server receives the request and analyzes its content using a natural language processing engine. This analysis utilizes common natural language processing libraries and services (e.g., open-source natural language toolkits or commercial cloud-based solutions).

[0178] As part of its analysis, the server uses emotion recognition technology to recognize the user's emotional state. This emotional state helps determine whether the user is experiencing emotions such as "anticipation" or "excitement." This emotional information is collected from a database along with the user's past purchase history and preference data.

[0179] Based on this information, the server uses a generative AI model to generate product suggestions tailored to the user. Specifically, this AI model leverages the latest, user-friendly, and flexible generative technologies to rank products based on text and scores.

[0180] Furthermore, when a user uploads visual data, such as images or videos, from their device, the server analyzes it and adjusts the suggestions to match the user's visual preferences. For this purpose, image processing libraries and advanced computer vision technologies (e.g., machine learning-based image classification tools) are used.

[0181] Users can provide feedback on the product suggestions presented to them, and the server receives this feedback to improve the suggestions. This feedback loop ensures that the system always provides product suggestions that meet user needs.

[0182] For example, when a user enters the prompt "I want a bag for everyday use," if the emotion engine determines that the user is feeling "excitement," stylish and brightly colored products will be prioritized for suggestion. This allows users to efficiently find the products they are looking for, thereby improving their satisfaction with the purchasing experience.

[0183] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0184] Step 1:

[0185] Users use their devices to send requests for goods and services in natural language through a chat box or voice input interface on the e-commerce platform. The input is received as text or voice data on the device and sent to the server over the network. For example, a user might type or voice the sentence "I want new sneakers." This input becomes the data sent to the server.

[0186] Step 2:

[0187] The server receives natural language input from the user. Using a natural language processing engine, the server analyzes the input data and extracts the requested product category and the user's intent. The input, received as text data, is passed to a language analysis system, where key words and phrases are extracted. As a result, categories such as sneakers and sports shoes are output.

[0188] Step 3:

[0189] The server uses an emotion engine to recognize the user's emotional state from the analyzed input. It analyzes text and voice tone as input data to obtain emotional information such as positive or negative. Specifically, the system determines the emotion "expectation" from the input "I want new sneakers" and outputs emotional information.

[0190] Step 4:

[0191] Based on the intent and sentiment information obtained in the previous step, the server collects and analyzes the user's purchase history and preference data from the database. Based on the acquired past purchase information, data calculations are performed to select highly relevant products. Frequently purchased color palettes and brands are identified from the shopping history and output.

[0192] Step 5:

[0193] The server uses a generative AI model to generate optimal product recommendations. Based on emotional information and purchase history, it outputs a list of suggested products. The generative AI model forms a group of products that it deems optimal for the user, based on past data and algorithms. Specifically, the AI ​​model considers the emotion of "expectation" and generates a list that prioritizes the latest sneaker models.

[0194] Step 6:

[0195] The server analyzes visual information from the user, including images and videos if available, and incorporates that information into the suggested products. Image files uploaded by the user are provided as input, and color and design patterns are extracted using image analysis technology. As a result, a product list matching the user's visual preferences is output and adjusted.

[0196] Step 7:

[0197] The server sends the final generated product suggestions to the user's terminal, where the user views the suggestions and provides feedback. The server analyzes the user's ratings and comments, and adjusts the suggestions if necessary. This feedback is used to improve the system in the next iteration.

[0198] (Application Example 2)

[0199] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0200] On e-commerce platforms, it is not easy for users to find products that suit their individual needs from a wide range of options, which can result in an unsatisfactory purchasing experience. Furthermore, traditional systems lack personalization that takes user emotions into account, resulting in a failure to provide suggestions that match the user's psychological state.

[0201] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0202] In this invention, the server includes means for receiving and analyzing requests in natural language from the user; means for collecting and analyzing the user's past purchase history and preference data based on the received requests; means for generating optimal product suggestions using generative artificial intelligence; means for customizing the suggestion content using multimodal data; means for presenting the generated product suggestions to the user, receiving feedback and responding accordingly; and means for recognizing the user's emotional state and adjusting the product suggestions based on those emotions. This enables personalized product suggestions that respond to the user's emotions.

[0203] "Natural language" refers to the system of words and sentences that humans use on a daily basis, and in this invention, it is a means used to interpret user requests.

[0204] "Means for analysis" refers to a system that has the processing capability to identify, classify, and understand input information based on its meaning and intent, and in this invention, it is used to decipher the user's requests.

[0205] "Purchase history" refers to a record of products and services that a user has purchased in the past. This information is used to identify the user's preferences and provide personalized recommendations.

[0206] "Preference data" refers to data that shows a user's preferences and tendencies, and is used to customize suggestions based on past behavior and choices.

[0207] "Generative artificial intelligence" refers to algorithms and models generated to solve problems according to specific purposes, and in this invention, it is used to create optimal product suggestions.

[0208] "Multimodal data" refers to data in multiple different formats (e.g., images, audio, text) and is used to enrich the proposed content in order to respond to user requests from multiple perspectives.

[0209] "Emotional state" refers to information that represents the user's psychological and emotional tendencies and circumstances, and is used to optimize the suggested content to match the user's emotions.

[0210] "Feedback" refers to user reactions and evaluations, and is information incorporated to improve product offerings.

[0211] The system used to realize this application example includes an e-commerce platform to which users connect using devices such as smartphones and computers. A server performs various processes on this platform.

[0212] First, the user sends their purchase request from their device to the server using natural language. The server receives this input and uses a natural language processing engine, such as Google Cloud Natural Language API, to analyze the user's request and understand its intent. The information analyzed by the server is integrated with the user's purchase history and preference data. In addition, sentiment recognition modules such as TensorFlow are used to determine the user's emotional state.

[0213] Next, generative artificial intelligence is used to generate product suggestions that best suit the user's requests and emotions. This process employs a generative AI model such as OpenAI, which determines the suggestions by referring to the user's individual prompt text. For example, a prompt text such as "Female in her 30s, wants a new spring bag, expresses excitement, has a history of purchasing imported goods. Please suggest three recommended products" is supplied to the model, and the generative AI constructs a specific product list.

[0214] Furthermore, the server performs multimodal data analysis to identify visual preferences from images and videos uploaded by the user and adjusts product recommendations accordingly. Image recognition APIs are used in this activity. Finally, the optimized product recommendations are delivered to the user's device. The user can view the presented products and provide feedback based on their preferences. The server uses this feedback to further improve future recommendations and increase user satisfaction.

[0215] This system allows users in e-commerce to experience more appropriate and appealing product suggestions tailored to their emotions and preferences.

[0216] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0217] Step 1:

[0218] A user accesses the e-commerce platform using their device and enters their purchase request in natural language. This input is sent to the server as text data. This input forms the basis for subsequent processing.

[0219] Step 2:

[0220] The server analyzes natural language requests received using the Google Cloud Natural Language API. Through this analysis, it identifies the user's intent and request content, and extracts keywords and categories related to the request. The output is structured data representing the user's request.

[0221] Step 3:

[0222] The server uses TensorFlow to perform emotion recognition from user input. It analyzes the input natural language data to identify specific emotional states such as joy, surprise, and sadness. The output is data representing the user's emotional state.

[0223] Step 4:

[0224] The server collects the user's past purchase history and preference data from the database and analyzes it in combination with the requests and emotional states extracted in the previous step. This process integrates data to gain a deeper understanding of the user's individual needs.

[0225] Step 5:

[0226] The server uses OpenAI's generative AI model to generate product suggestions based on the user's requests and emotions. It provides prompts to the generative AI model, giving specific instructions such as, "Female, in her 30s, wants a new spring bag, expresses excitement. Has a history of purchasing imported goods. Please suggest three recommended items." The output is a list of customized product suggestions.

[0227] Step 6:

[0228] The server uses an image recognition API to analyze images and video data uploaded by users and identify their visual preferences. This analysis allows the server to adjust product recommendations based on visual data. The output is a filtered list of products corresponding to the user's visual preferences.

[0229] Step 7:

[0230] Finally, the server integrates product recommendations based on the analysis and generated results, and transmits a personalized list to the user's device. The user can view this list and provide feedback. This feedback is sent to the server and used to further optimize the recommendations.

[0231] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0232] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0233] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0234] [Second Embodiment]

[0235] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0236] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0237] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0238] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0239] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0240] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0241] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0242] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0243] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0244] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0245] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0246] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0247] The system of this invention is designed to be integrated into an e-commerce platform and to provide personalized product recommendations to users. A specific example of its operation is shown below.

[0248] First, the user accesses an e-commerce site through their device and sends an inquiry in natural language about the product or service they are considering purchasing. This operation is typically performed via a chat box or voice input interface.

[0249] When the server receives a request from a user, it first passes it to a natural language processing engine. Here, the user's inquiry is analyzed to clarify the intent and content of the request. For example, if a user enters "I want running shoes," the server extracts the keywords "running" and "shoes" and focuses on the sports shoes category.

[0250] Next, the server accesses the database to collect the user's past purchase history and preference data. This allows it to identify what products the user has previously purchased, as well as their preferred brands and styles. Based on this information, the generative artificial intelligence selects and suggests products that it believes are best suited to the user.

[0251] Furthermore, users can upload additional information using their devices, such as images and videos related to their purchases. The server analyzes this multimodal data and customizes product recommendations based on the user's visual preferences.

[0252] After product suggestions are generated, the server presents them to the user's terminal. The user has the opportunity to view the suggested product list and provide feedback. For example, if the user requests that "the design should be more colorful," the server uses the generating AI to re-evaluate the suggestions and present new, more appropriate products.

[0253] In this way, the system aims to provide product suggestions that more accurately meet user needs and improve the purchasing experience. Through this series of processes, the interaction between users and the e-commerce platform becomes more efficient and satisfying.

[0254] The following describes the processing flow.

[0255] Step 1:

[0256] Users access e-commerce platforms using their devices and enter requests for desired products or services in natural language. This is typically done through the site's chat window.

[0257] Step 2:

[0258] The server receives a request from the user. This information is sent to a natural language processing engine, which analyzes the request and clarifies its content and intent.

[0259] Step 3:

[0260] Based on the analyzed information, the server retrieves the user's past purchase history and preference data from a database. This enables product recommendations that take into account the user's preferences and past behavior.

[0261] Step 4:

[0262] The server uses artificial intelligence to integrate user requests with historical data to generate an optimal product list. This list includes recommended products and similar items.

[0263] Step 5:

[0264] Users upload images and videos related to products using their devices. This multimodal data is sent to the server as additional preference information.

[0265] Step 6:

[0266] The server analyzes multimodal data to identify the user's visual preferences. Based on this information, it customizes the suggested product list to create more specific product recommendations.

[0267] Step 7:

[0268] The server then displays the final product suggestions generated by the server to the user's terminal. The user can browse these products and provide further requests or feedback.

[0269] Step 8:

[0270] The server receives user feedback, uses AI to re-evaluate the suggested products based on that feedback, and then presents a further refined product list to provide suggestions that best meet the user's needs.

[0271] (Example 1)

[0272] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0273] Modern e-commerce platforms offer generic and uniform product suggestions to users, failing to adequately reflect each user's individual preferences and past purchase history. Therefore, there is a need to develop systems that can satisfy users' purchasing desires and assist them in making optimal product choices.

[0274] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0275] In this invention, the server includes means for receiving and analyzing requests from users in natural language, means for collecting and analyzing the user's past purchase history and preference information based on the received requests, and means for generating optimal product suggestions using generative artificial intelligence. This enables personalized product suggestions for each user and improves the purchasing experience.

[0276] "Requests made in natural language" refer to sentences or phrases that users use to communicate their intentions in text or voice, without being constrained by any specific format.

[0277] "Purchase history" refers to detailed information and records of products that a user has purchased in the past.

[0278] "Preference information" refers to data indicating a user's preferences and tendencies, and refers to personal judgment criteria that affect product selection.

[0279] "Generative artificial intelligence" refers to an algorithm or system that has the ability to generate product proposals suitable for the user using machine learning technology.

[0280] "Diverse media data" refers to data in different formats, including visual information such as images and videos and auditory information such as voices.

[0281] "Item proposal" refers to a list of products and services recommended based on the needs and preferences of the user.

[0282] "Opinion" refers to the feelings, requests, and hopes expressed by the user regarding the proposed product.

[0283] The embodiments for implementing the present invention will be described. This system realizes advanced processing for making personalized product proposals to the user. The specific operations are shown below.

[0284] First, the user uses a terminal to access an e-commerce platform and then sends a natural language inquiry about the desired product or service of interest. Natural language inquiries are usually made through a chat box or a voice input interface.

[0285] Next, the server receives the user's request and first analyzes it using a natural language processing (NLP) engine. As a result of the analysis, the user's intention and request content are clarified. Based on this information, when the user wants "running shoes", the server extracts the keywords "running" and "shoes" and identifies the category of the product proposal.

[0286] Subsequently, the server accesses the internal database and collects data based on the user's past purchase history and preference information. This enables the identification of the products, brands, and styles that the user has purchased in the past. These data serve as important fundamental information for proposing the most suitable products to the user.

[0287] After that, the server uses the generative AI model to generate optimal product recommendations for the user based on this analyzed data. The generative AI model is based on machine learning algorithms and lists the most suitable products while taking into account the user's past preferences.

[0288] Furthermore, the user can upload images and videos related to purchases through the terminal. The server analyzes this multimodal data and further customizes product recommendations based on the user's visual preferences. In this process, preferences for colors and designs, in particular, are important elements.

[0289] Finally, the server presents the generated product recommendations to the user's terminal. The user has the opportunity to view the presented product list and provide feedback. This feedback can be used to further improve the recommendations.

[0290] As a specific example, when the user inputs "I want a tent for camping", the server refers to the past purchase history of outdoor supplies based on the keywords "camping" and "tent" and proposes appropriate products. An example of a prompt sentence for the generative AI model is "Please propose new products related to the camping supplies that the user has bought in the past".

[0291] Thus, this system accurately meets user needs and improves the purchasing experience.

[0292] The flow of the specific process in Example 1 will be described using FIG. 11.

[0293] Step 1:

[0294] Users use their devices to access e-commerce platforms and submit inquiries in natural language about products and services they are considering purchasing. This input can be in text or voice format and is sent from the device to the server. For example, a user might ask, "I want running shoes."

[0295] Step 2:

[0296] The server receives requests from users in natural language and passes them to a natural language processing (NLP) engine for analysis. The input text is extracted through analysis into keywords such as "running" and "shoes." This output clarifies the user's intent and request.

[0297] Step 3:

[0298] Based on user requests, the server accesses a database within the system to collect the user's past purchase history and preference information. Database queries are sent as input, the data is processed, and the output includes data on the user's preferences and past purchases. Specifically, this includes information on previously purchased sports shoes and brands.

[0299] Step 4:

[0300] The server utilizes a generative AI model to generate optimal product suggestions based on the user's past data and received requests. The AI ​​model uses machine learning algorithms to analyze a large amount of user data and suggest new products. The input for this step is past preference data and product category information, and the output is a personalized list of products.

[0301] Step 5:

[0302] Users can upload images and videos related to purchases using a terminal. The server receives these and analyzes them as multimodal data. Visual data as input is analyzed to identify the user's visual preferences and reflected in optimal proposals. For example, if the user prefers colorful designs, that information is taken into account in product proposals.

[0303] Step 6:

[0304] The server presents customized product proposals to the user terminal. Using the product list generated as input, a prompt sentence is used to present relevant information to the user. As output, the user receives the product list and can examine the details. The user can provide feedback there and return data to the server to further improve the accuracy of the proposal.

[0305] (Application Example 1)

[0306] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0307] In online shopping, it is difficult to quickly and appropriately provide products that meet diverse user needs. In this situation, it is hard to say that product proposals based simply on the user's past purchase history and basic information are sufficient, and more advanced analysis and personalization are required. Also, it is essential to flexibly update the proposal content by leveraging the user's feedback. Furthermore, the ability to accurately grasp the user's needs by using natural language input and multimodal data is also important. Solving such problems is required.

[0308] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0309] In this invention, the server includes means for receiving and analyzing requests in natural language from users, means for collecting and analyzing the user's past purchase history and preference data, means for generating optimal product suggestions using generative artificial intelligence, and means for providing customized product suggestions using machine learning models. This enables rapid and highly accurate product suggestions to meet the diverse needs of users.

[0310] "Requests in natural language" refer to requests or questions that users input using everyday language.

[0311] "Means of analysis" refers to a system or program that has the function of breaking down received information and performing a process to extract its meaning and intent.

[0312] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0313] "Preference data" refers to information about preferences and trends collected based on a user's past preferences, interests, and usage history.

[0314] "Generative artificial intelligence" refers to machine learning techniques and algorithms that dynamically generate optimal suggestions and answers based on user requests and preferences.

[0315] "Multimodal data" refers to information that combines multiple types of data, such as audio, images, and text.

[0316] "Means of customization" refers to methods and technologies for adjusting and optimizing output according to the individual user's needs and preferences.

[0317] "Feedback" refers to the information provided by users regarding their user experience and suggestions, and is used to improve the system's performance.

[0318] "Natural language processing capabilities" refer to the technology that enables computers to understand and process human language.

[0319] A "data storage structure" refers to a system of databases and file systems designed to efficiently store and utilize information.

[0320] A "machine learning model" refers to an algorithm that learns from large amounts of data and recognizes patterns to make future predictions and classifications.

[0321] In order to implement this invention, it is mainly necessary for a server and a user terminal to work together. The server has advanced computing resources and is responsible for processing user requests and generating appropriate product suggestions. User terminals refer to various devices such as smartphones and computers, and are means for users to interface with the system.

[0322] When a server receives a request from a user in natural language, it first uses a natural language processing engine to analyze the request. Typically, machine learning-based natural language processing technologies, such as OpenAI's API, are used for this process. Based on the analysis results, the server identifies the user's intent and emotions regarding the request.

[0323] Next, the server retrieves the user's past purchase history and preference data from the database. This data is managed by a database management system such as SQLite. Once the information is collected, the server uses machine learning models, such as TensorFlow or PyTorch, to create generative AI models and make optimal product recommendations.

[0324] Furthermore, if multimodal data such as images and videos are available from the user, this data is used to further customize the suggestions. At this time, computer vision technology and deep learning are used to analyze the user's visual preferences and generate new feedback.

[0325] The generated product suggestions are sent to the user's device, where the user evaluates them. If feedback is submitted, for example, if the user suggests "a more colorful design would be better," the server re-evaluates the suggestion and updates it. In this way, the system can continuously provide the best suggestions that meet the user's needs.

[0326] As a concrete example, the following is an example of a prompt message that analyzes the request, "I want lightweight, waterproof running shoes."

[0327] Example of a prompt:

[0328] "Analyze the user's query: 'I want lightweight, waterproof running shoes,' and identify the requested product category and characteristics."

[0329] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0330] Step 1:

[0331] The user uses a device to input a request using natural language, either by voice or text. This request clearly indicates the user's purchasing interest and is specific, such as "I want lightweight, waterproof running shoes." The device then sends this request to the server.

[0332] Step 2:

[0333] The server receives requests written in natural language and analyzes them using a natural language processing engine. This analysis breaks down the user's request into keywords, extracting information such as "lightweight," "waterproof," and "running shoes." The input is the user's request, and the output is a set of analyzed keywords.

[0334] Step 3:

[0335] The server then retrieves and aggregates user purchase history and preference data from the database. At this stage, a database management system such as SQLite is used. The input is the user's ID, and the output is the user's past purchase history and preference data. This information is used to process the data and analyze user interests and preferences.

[0336] Step 4:

[0337] The server uses a machine learning model to generate optimal product suggestions based on the analysis results and user preference data. Here, a generative AI model is used to generate a list of products best suited to satisfy the user's requirements. The input is the output data from steps 2 and 3, and the output is a list of product suggestions.

[0338] Step 5:

[0339] If the user submits additional images or videos, this data is analyzed and multimodal data is used to further customize product suggestions. Image processing techniques are used to analyze the user's visual preferences. The input is image or video data from the user, and the output is a list of more customized product suggestions.

[0340] Step 6:

[0341] The generated product suggestions are sent to the user's terminal, where the user views the suggested products and provides feedback. The server receives the feedback, re-evaluates them, and generates suggestions again if necessary. The input is the user's feedback, and the output is the revised product suggestions.

[0342] Step 7:

[0343] The server performs a process based on this feedback, updating the generating AI model to improve the accuracy of future suggestions. The server constantly learns from new data to improve the performance of product suggestions.

[0344] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0345] The system of the present invention is designed to provide personalized product recommendations to users on an e-commerce platform, and aims to provide a more advanced user experience by incorporating emotion recognition technology.

[0346] Users first use their device to inquire about the products or services they wish to purchase using natural language via a chat box or voice input interface on the e-commerce platform.

[0347] The server receives inquiries from users and uses a natural language processing engine to analyze the content and intent of the requests. Simultaneously, an emotion engine recognizes the user's emotional state from their input, identifying emotions such as joy, surprise, and sadness. This allows the server to understand what emotions are behind the user's requests.

[0348] Based on the analyzed requests and emotional states, the server collects the user's past purchase history and preference data from a database, integrates and analyzes this information, and then applies emotional information to customize the recommendations.

[0349] Furthermore, it utilizes generative artificial intelligence to generate optimal product recommendations. Here, the emotional state identified by the emotion engine is used to determine the priority of product recommendations and to highlight specific product groups. For example, if the user expresses the emotion of "surprise," unique and new products will be prioritized in the recommendations.

[0350] The generated product suggestions are further customized by incorporating multimodal data. If a user uploads images or videos from their device, that data is analyzed, and the suggestions are adjusted to match the user's visual preferences.

[0351] Finally, the server presents the generated suggestions to the user's terminal. The user can view the suggested products and provide additional feedback. The server incorporates this feedback and adjusts the suggestions as needed to provide an experience that recommends the most appropriate products to meet the user's needs. For example, if a user requests "I want a bag for everyday use," and the emotion engine determines that the user is experiencing "enjoyment," then stylishly designed and colorful products will be prioritized in the suggestions.

[0352] The following describes the processing flow.

[0353] Step 1:

[0354] Users access e-commerce platforms using their devices, inputting the products or services they wish to purchase in natural language, and submitting inquiries. Chat windows and voice input are used.

[0355] Step 2:

[0356] The server receives a request written in natural language from the user. This request is then passed to a natural language processing engine for analysis.

[0357] Step 3:

[0358] The server uses a natural language processing engine to analyze the user's request, clarifying its intent and requirements. This analysis identifies key keywords and desired product categories.

[0359] Step 4:

[0360] The server uses an emotion engine to recognize the emotional state contained in the user's natural language request. Emotions such as joy, surprise, and sadness are identified, and the user's mental state is recorded.

[0361] Step 5:

[0362] The server retrieves the user's past purchase history and preference data from the database. This data reflects the user's preferences and past purchasing behavior and can be used to make recommendations.

[0363] Step 6:

[0364] The server integrates historical data collected with emotional information obtained from the emotion engine, and uses generative artificial intelligence to generate optimal product suggestions. Here, the suggested products are adjusted according to the user's emotional state.

[0365] Step 7:

[0366] Users can upload images and videos related to products from their devices. This multimodal data is sent to the server to complement the user's visual preferences.

[0367] Step 8:

[0368] The server analyzes multimodal data to identify the user's visual preferences and then customizes the suggestions accordingly. This makes the product list more closely match the user's specific desires.

[0369] Step 9:

[0370] The server then displays the final product suggestions to the user's terminal. The user can view the suggested products and provide further feedback.

[0371] Step 10:

[0372] The server receives user feedback, and based on that feedback, the AI ​​generates suggestions to re-evaluate and adjust them, continuously presenting the product that best suits the user's needs.

[0373] (Example 2)

[0374] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0375] In modern e-commerce platforms, personalized product recommendations are crucial. However, simply basing recommendations on past purchase history and preferences fails to adequately reflect users' actual needs and emotions, limiting their ability to improve satisfaction. Furthermore, when suggesting appropriate products based on user input, emotional states are often overlooked, making it difficult to provide an optimal purchasing experience.

[0376] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0377] In this invention, the server includes means for receiving natural language input from the user and analyzing its content, means for collecting and analyzing information on the user's purchase history and preferences based on the received input, and means for generating appropriate product suggestions using artificial intelligence. This makes it possible to provide more accurate and personalized product suggestions that also take into account the user's emotional information.

[0378] "User-generated natural language input" refers to users conveying information to an electronic platform using expressions from everyday conversation.

[0379] "Means for analyzing content" refers to a device or software that analyzes received natural language input and processes it to understand its meaning and intent.

[0380] "Purchase history and preference information" refers to data related to products and services that a user has previously acquired, as well as their preferences for those products and services.

[0381] "Generative artificial intelligence" refers to an intelligent program or algorithm that automatically generates optimal product and service suggestions based on large amounts of data.

[0382] "Data in diverse formats" refers to a collection of data that encompasses information obtained in different media formats, such as text, images, audio, and video.

[0383] "Visual preference" refers to a user's particular preferences regarding visual elements such as colors, designs, and styles.

[0384] "Emotional information" refers to information that indicates an emotional state, inferred from user input and interactions.

[0385] The present invention provides a system that generates personalized product suggestions when a user purchases goods or services on an e-commerce platform using a terminal, by having the user input requests in natural language and having the server analyze and process those inputs.

[0386] Users can make product requests via text or voice through their device. If a user enters a request such as "I want new sneakers," the server receives the request and analyzes its content using a natural language processing engine. This analysis utilizes common natural language processing libraries and services (e.g., open-source natural language toolkits or commercial cloud-based solutions).

[0387] As part of its analysis, the server uses emotion recognition technology to recognize the user's emotional state. This emotional state helps determine whether the user is experiencing emotions such as "anticipation" or "excitement." This emotional information is collected from a database along with the user's past purchase history and preference data.

[0388] Based on this information, the server uses a generative AI model to generate product suggestions tailored to the user. Specifically, this AI model leverages the latest, user-friendly, and flexible generative technologies to rank products based on text and scores.

[0389] Furthermore, when a user uploads visual data, such as images or videos, from their device, the server analyzes it and adjusts the suggestions to match the user's visual preferences. For this purpose, image processing libraries and advanced computer vision technologies (e.g., machine learning-based image classification tools) are used.

[0390] Users can provide feedback on the product suggestions presented to them, and the server receives this feedback to improve the suggestions. This feedback loop ensures that the system always provides product suggestions that meet user needs.

[0391] For example, when a user enters the prompt "I want a bag for everyday use," if the emotion engine determines that the user is feeling "excitement," stylish and brightly colored products will be prioritized for suggestion. This allows users to efficiently find the products they are looking for, thereby improving their satisfaction with the purchasing experience.

[0392] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0393] Step 1:

[0394] Users use their devices to send requests for goods and services in natural language through a chat box or voice input interface on the e-commerce platform. The input is received as text or voice data on the device and sent to the server over the network. For example, a user might type or voice the sentence "I want new sneakers." This input becomes the data sent to the server.

[0395] Step 2:

[0396] The server receives natural language input from the user. Using a natural language processing engine, the server analyzes the input data and extracts the requested product category and the user's intent. The input, received as text data, is passed to a language analysis system, where key words and phrases are extracted. As a result, categories such as sneakers and sports shoes are output.

[0397] Step 3:

[0398] The server uses an emotion engine to recognize the user's emotional state from the analyzed input. It analyzes text and voice tone as input data to obtain emotional information such as positive or negative. Specifically, the system determines the emotion "expectation" from the input "I want new sneakers" and outputs emotional information.

[0399] Step 4:

[0400] Based on the intent and sentiment information obtained in the previous step, the server collects and analyzes the user's purchase history and preference data from the database. Based on the acquired past purchase information, data calculations are performed to select highly relevant products. Frequently purchased color palettes and brands are identified from the shopping history and output.

[0401] Step 5:

[0402] The server uses a generative AI model to generate optimal product recommendations. Based on emotional information and purchase history, it outputs a list of suggested products. The generative AI model forms a group of products that it deems optimal for the user, based on past data and algorithms. Specifically, the AI ​​model considers the emotion of "expectation" and generates a list that prioritizes the latest sneaker models.

[0403] Step 6:

[0404] The server analyzes visual information from the user, including images and videos if available, and incorporates that information into the suggested products. Image files uploaded by the user are provided as input, and color and design patterns are extracted using image analysis technology. As a result, a product list matching the user's visual preferences is output and adjusted.

[0405] Step 7:

[0406] The server sends the final generated product suggestions to the user's terminal, where the user views the suggestions and provides feedback. The server analyzes the user's ratings and comments, and adjusts the suggestions if necessary. This feedback is used to improve the system in the next iteration.

[0407] (Application Example 2)

[0408] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0409] On e-commerce platforms, it is not easy for users to find products that suit their individual needs from a wide range of options, which can result in an unsatisfactory purchasing experience. Furthermore, traditional systems lack personalization that takes user emotions into account, resulting in a failure to provide suggestions that match the user's psychological state.

[0410] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0411] In this invention, the server includes means for receiving and analyzing requests in natural language from the user; means for collecting and analyzing the user's past purchase history and preference data based on the received requests; means for generating optimal product suggestions using generative artificial intelligence; means for customizing the suggestion content using multimodal data; means for presenting the generated product suggestions to the user, receiving feedback and responding accordingly; and means for recognizing the user's emotional state and adjusting the product suggestions based on those emotions. This enables personalized product suggestions that respond to the user's emotions.

[0412] "Natural language" refers to the system of words and sentences that humans use on a daily basis, and in this invention, it is a means used to interpret user requests.

[0413] "Means for analysis" refers to a system that has the processing capability to identify, classify, and understand input information based on its meaning and intent, and in this invention, it is used to decipher the user's requests.

[0414] "Purchase history" refers to a record of products and services that a user has purchased in the past. This information is used to identify the user's preferences and provide personalized recommendations.

[0415] "Preference data" refers to data that shows a user's preferences and tendencies, and is used to customize suggestions based on past behavior and choices.

[0416] "Generative artificial intelligence" refers to algorithms and models generated to solve problems according to specific purposes, and in this invention, it is used to create optimal product suggestions.

[0417] "Multimodal data" refers to data in multiple different formats (e.g., images, audio, text) and is used to enrich the proposed content in order to respond to user requests from multiple perspectives.

[0418] "Emotional state" refers to information that represents the user's psychological and emotional tendencies and circumstances, and is used to optimize the suggested content to match the user's emotions.

[0419] "Feedback" refers to user reactions and evaluations, and is information incorporated to improve product offerings.

[0420] The system used to realize this application example includes an e-commerce platform to which users connect using devices such as smartphones and computers. A server performs various processes on this platform.

[0421] First, the user sends their purchase request from their device to the server using natural language. The server receives this input and uses a natural language processing engine, such as the Google Cloud Natural Language API, to analyze the user's request and understand its intent. The information analyzed by the server is integrated with the user's purchase history and preference data. Additionally, sentiment recognition modules such as TensorFlow are used to determine the user's emotional state.

[0422] Next, generative artificial intelligence is used to generate product suggestions that best suit the user's requests and emotions. This process employs a generative AI model such as OpenAI, which determines the suggestions by referring to the user's individual prompt text. For example, a prompt text such as "Female in her 30s, wants a new spring bag, expresses excitement, has a history of purchasing imported goods. Please suggest three recommended products" is supplied to the model, and the generative AI constructs a specific product list.

[0423] Furthermore, the server performs multimodal data analysis to identify visual preferences from images and videos uploaded by the user and adjusts product recommendations accordingly. Image recognition APIs are used in this activity. Finally, the optimized product recommendations are delivered to the user's device. The user can view the presented products and provide feedback based on their preferences. The server uses this feedback to further improve future recommendations and increase user satisfaction.

[0424] This system allows users in e-commerce to experience more appropriate and appealing product suggestions tailored to their emotions and preferences.

[0425] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0426] Step 1:

[0427] A user accesses the e-commerce platform using their device and enters their purchase request in natural language. This input is sent to the server as text data. This input forms the basis for subsequent processing.

[0428] Step 2:

[0429] The server analyzes natural language requests received using the Google Cloud Natural Language API. Through this analysis, it identifies the user's intent and request content, and extracts keywords and categories related to the request. The output is structured data representing the user's request.

[0430] Step 3:

[0431] The server uses TensorFlow to perform emotion recognition from user input. It analyzes the input natural language data to identify specific emotional states such as joy, surprise, and sadness. The output is data representing the user's emotional state.

[0432] Step 4:

[0433] The server collects the user's past purchase history and preference data from the database and analyzes it in combination with the requests and emotional states extracted in the previous step. This process integrates data to gain a deeper understanding of the user's individual needs.

[0434] Step 5:

[0435] The server uses OpenAI's generative AI model to generate product suggestions based on the user's requests and emotions. It provides prompts to the generative AI model, giving specific instructions such as, "Female, in her 30s, wants a new spring bag, expresses excitement. Has a history of purchasing imported goods. Please suggest three recommended items." The output is a list of customized product suggestions.

[0436] Step 6:

[0437] The server uses an image recognition API to analyze images and video data uploaded by users and identify their visual preferences. This analysis allows the server to adjust product recommendations based on visual data. The output is a filtered list of products corresponding to the user's visual preferences.

[0438] Step 7:

[0439] Finally, the server integrates product recommendations based on the analysis and generated results, and transmits a personalized list to the user's device. The user can view this list and provide feedback. This feedback is sent to the server and used to further optimize the recommendations.

[0440] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0441] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0442] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0443] [Third Embodiment]

[0444] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0445] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0446] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0447] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0448] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0449] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0450] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0451] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0452] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0453] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0454] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0455] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0456] The system of this invention is designed to be integrated into an e-commerce platform and to provide personalized product recommendations to users. A specific example of its operation is shown below.

[0457] First, the user accesses an e-commerce site through their device and sends an inquiry in natural language about the product or service they are considering purchasing. This operation is typically performed via a chat box or voice input interface.

[0458] When the server receives a request from a user, it first passes it to a natural language processing engine. Here, the user's inquiry is analyzed to clarify the intent and content of the request. For example, if a user enters "I want running shoes," the server extracts the keywords "running" and "shoes" and focuses on the sports shoes category.

[0459] Next, the server accesses the database to collect the user's past purchase history and preference data. This allows it to identify what products the user has previously purchased, as well as their preferred brands and styles. Based on this information, the generative artificial intelligence selects and suggests products that it believes are best suited to the user.

[0460] Furthermore, users can upload additional information using their devices, such as images and videos related to their purchases. The server analyzes this multimodal data and customizes product recommendations based on the user's visual preferences.

[0461] After product suggestions are generated, the server presents them to the user's terminal. The user has the opportunity to view the suggested product list and provide feedback. For example, if the user requests that "the design should be more colorful," the server uses the generating AI to re-evaluate the suggestions and present new, more appropriate products.

[0462] In this way, the system aims to provide product suggestions that more accurately meet user needs and improve the purchasing experience. Through this series of processes, the interaction between users and the e-commerce platform becomes more efficient and satisfying.

[0463] The following describes the processing flow.

[0464] Step 1:

[0465] Users access e-commerce platforms using their devices and enter requests for desired products or services in natural language. This is typically done through the site's chat window.

[0466] Step 2:

[0467] The server receives a request from the user. This information is sent to a natural language processing engine, which analyzes the request and clarifies its content and intent.

[0468] Step 3:

[0469] Based on the analyzed information, the server retrieves the user's past purchase history and preference data from a database. This enables product recommendations that take into account the user's preferences and past behavior.

[0470] Step 4:

[0471] The server uses artificial intelligence to integrate user requests with historical data to generate an optimal product list. This list includes recommended products and similar items.

[0472] Step 5:

[0473] Users upload images and videos related to products using their devices. This multimodal data is sent to the server as additional preference information.

[0474] Step 6:

[0475] The server analyzes multimodal data to identify the user's visual preferences. Based on this information, it customizes the suggested product list to create more specific product recommendations.

[0476] Step 7:

[0477] The server then displays the final product suggestions generated by the server to the user's terminal. The user can browse these products and provide further requests or feedback.

[0478] Step 8:

[0479] The server receives user feedback, uses AI to re-evaluate the suggested products based on that feedback, and then presents a further refined product list to provide suggestions that best meet the user's needs.

[0480] (Example 1)

[0481] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0482] Modern e-commerce platforms offer generic and uniform product suggestions to users, failing to adequately reflect each user's individual preferences and past purchase history. Therefore, there is a need to develop systems that can satisfy users' purchasing desires and assist them in making optimal product choices.

[0483] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0484] In this invention, the server includes means for receiving and analyzing requests from users in natural language, means for collecting and analyzing the user's past purchase history and preference information based on the received requests, and means for generating optimal product suggestions using generative artificial intelligence. This enables personalized product suggestions for each user and improves the purchasing experience.

[0485] "Requests made in natural language" refer to sentences or phrases that users use to communicate their intentions in text or voice, without being constrained by any specific format.

[0486] "Purchase history" refers to detailed information and records of products that a user has purchased in the past.

[0487] "Preference information" refers to data that shows a user's preferences and tendencies, and represents personal judgment criteria that influence product selection.

[0488] "Generative artificial intelligence" refers to an algorithm or system that has the ability to generate product suggestions tailored to the user using machine learning techniques.

[0489] "Diverse media data" refers to data in different formats, including visual information such as images and videos, and auditory information such as audio.

[0490] "Product recommendations" refer to a list of products and services recommended based on the user's needs and preferences.

[0491] "Opinions" refer to the thoughts, requests, and wishes that users express regarding the proposed product.

[0492] The embodiments for carrying out the present invention will now be described. This system implements advanced processing for providing personalized product recommendations to users. Its specific operation is shown below.

[0493] First, users use their devices to access e-commerce platforms and then submit inquiries in natural language about desired products or services they are interested in. These natural language inquiries are typically made through chat boxes or voice input interfaces.

[0494] Next, the server receives the user's request and first analyzes it using a natural language processing (NLP) engine. The analysis clarifies the user's intent and request. Based on this information, if the user says "I want running shoes," the server extracts the keywords "running" and "shoes" and identifies the product category to suggest.

[0495] Next, the server accesses an internal database to collect data based on the user's past purchase history and preferences. This allows the server to identify products the user has previously purchased, as well as their preferred brands and styles. This data serves as crucial foundational information for suggesting the most suitable products to the user.

[0496] Subsequently, the server uses a generative AI model to generate optimal product recommendations for the user based on this analytical data. The generative AI model is based on machine learning algorithms and lists the most suitable products while taking into account the user's past preferences.

[0497] Furthermore, users can upload images and videos related to their purchases through their devices. The server analyzes this multimodal data and further customizes product recommendations based on the user's visual preferences. In this process, preferences for color and design are particularly important factors.

[0498] Finally, the server displays the generated product suggestions to the user's device. The user has the opportunity to view the presented product list and provide feedback. This feedback will be used to further improve the suggestions.

[0499] For example, if a user enters "I want a tent I can use for camping," the server will refer to the user's past purchase history of outdoor equipment based on the keywords "camping" and "tent," and suggest appropriate products. An example of a prompt to the generating AI model is, "Please suggest new products related to the camping equipment the user has purchased in the past."

[0500] This will allow the system to accurately meet user needs and improve the purchasing experience.

[0501] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0502] Step 1:

[0503] Users use their devices to access e-commerce platforms and submit inquiries in natural language about products and services they are considering purchasing. This input can be in text or voice format and is sent from the device to the server. For example, a user might ask, "I want running shoes."

[0504] Step 2:

[0505] The server receives requests from users in natural language and passes them to a natural language processing (NLP) engine for analysis. The input text is extracted through analysis into keywords such as "running" and "shoes." This output clarifies the user's intent and request.

[0506] Step 3:

[0507] Based on user requests, the server accesses a database within the system to collect the user's past purchase history and preference information. Database queries are sent as input, the data is processed, and the output includes data on the user's preferences and past purchases. Specifically, this includes information on previously purchased sports shoes and brands.

[0508] Step 4:

[0509] The server utilizes a generative AI model to generate optimal product suggestions based on the user's past data and received requests. The AI ​​model uses machine learning algorithms to analyze a large amount of user data and suggest new products. The input for this step is past preference data and product category information, and the output is a personalized list of products.

[0510] Step 5:

[0511] Users can upload images and videos related to their purchases using their devices. The server receives these and analyzes them as multimodal data. The visual data used as input is analyzed to identify the user's visual preferences, which are then reflected in the most suitable recommendations. For example, if a user prefers colorful designs, that information is taken into account in the product recommendations.

[0512] Step 6:

[0513] The server presents customized product suggestions to the user's terminal. It uses a generated product list as input and employs prompts to present relevant information to the user. As output, the user receives the product list and can review the details. The user can then provide feedback and return data to the server to further improve the accuracy of the suggestions.

[0514] (Application Example 1)

[0515] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0516] Providing products that quickly and appropriately meet the diverse needs of users in online shopping is challenging. In this situation, simply suggesting products based on a user's past purchase history and basic information is insufficient; more advanced analysis and personalization are required. Furthermore, it is essential to flexibly update suggestions using user feedback. The ability to accurately grasp user needs using natural language input and multimodal data is also crucial. Solving these challenges is essential.

[0517] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0518] In this invention, the server includes means for receiving and analyzing requests in natural language from users, means for collecting and analyzing the user's past purchase history and preference data, means for generating optimal product suggestions using generative artificial intelligence, and means for providing customized product suggestions using machine learning models. This enables rapid and highly accurate product suggestions to meet the diverse needs of users.

[0519] "Requests in natural language" refer to requests or questions that users input using everyday language.

[0520] "Means of analysis" refers to a system or program that has the function of breaking down received information and performing a process to extract its meaning and intent.

[0521] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0522] "Preference data" refers to information about preferences and trends collected based on a user's past preferences, interests, and usage history.

[0523] "Generative artificial intelligence" refers to machine learning techniques and algorithms that dynamically generate optimal suggestions and answers based on user requests and preferences.

[0524] "Multimodal data" refers to information that combines multiple types of data, such as audio, images, and text.

[0525] "Means of customization" refers to methods and technologies for adjusting and optimizing output according to the individual user's needs and preferences.

[0526] "Feedback" refers to the information provided by users regarding their user experience and suggestions, and is used to improve the system's performance.

[0527] "Natural language processing capabilities" refer to the technology that enables computers to understand and process human language.

[0528] A "data storage structure" refers to a system of databases and file systems designed to efficiently store and utilize information.

[0529] A "machine learning model" refers to an algorithm that learns from large amounts of data and recognizes patterns to make future predictions and classifications.

[0530] In order to implement this invention, it is mainly necessary for a server and a user terminal to work together. The server has advanced computing resources and is responsible for processing user requests and generating appropriate product suggestions. User terminals refer to various devices such as smartphones and computers, and are means for users to interface with the system.

[0531] When a server receives a request from a user in natural language, it first uses a natural language processing engine to analyze the request. Typically, machine learning-based natural language processing technologies, such as OpenAI's API, are used for this process. Based on the analysis results, the server identifies the user's intent and emotions regarding the request.

[0532] Next, the server retrieves the user's past purchase history and preference data from the database. This data is managed by a database management system such as SQLite. Once the information is collected, the server uses machine learning models, such as TensorFlow or PyTorch, to create generative AI models and make optimal product recommendations.

[0533] Furthermore, if multimodal data such as images and videos are available from the user, this data is used to further customize the suggestions. At this time, computer vision technology and deep learning are used to analyze the user's visual preferences and generate new feedback.

[0534] The generated product suggestions are sent to the user's device, where the user evaluates them. If feedback is submitted, for example, if the user suggests "a more colorful design would be better," the server re-evaluates the suggestion and updates it. In this way, the system can continuously provide the best suggestions that meet the user's needs.

[0535] As a concrete example, the following is an example of a prompt message that analyzes the request, "I want lightweight, waterproof running shoes."

[0536] Example of a prompt:

[0537] "Analyze the user's query: 'I want lightweight, waterproof running shoes,' and identify the requested product category and characteristics."

[0538] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0539] Step 1:

[0540] The user uses a device to input a request using natural language, either by voice or text. This request clearly indicates the user's purchasing interest and is specific, such as "I want lightweight, waterproof running shoes." The device then sends this request to the server.

[0541] Step 2:

[0542] The server receives requests written in natural language and analyzes them using a natural language processing engine. This analysis breaks down the user's request into keywords, extracting information such as "lightweight," "waterproof," and "running shoes." The input is the user's request, and the output is a set of analyzed keywords.

[0543] Step 3:

[0544] The server then retrieves and aggregates user purchase history and preference data from the database. At this stage, a database management system such as SQLite is used. The input is the user's ID, and the output is the user's past purchase history and preference data. This information is used to process the data and analyze user interests and preferences.

[0545] Step 4:

[0546] The server uses a machine learning model to generate optimal product suggestions based on the analysis results and user preference data. Here, a generative AI model is used to generate a list of products best suited to satisfy the user's requirements. The input is the output data from steps 2 and 3, and the output is a list of product suggestions.

[0547] Step 5:

[0548] If the user submits additional images or videos, this data is analyzed and multimodal data is used to further customize product suggestions. Image processing techniques are used to analyze the user's visual preferences. The input is image or video data from the user, and the output is a list of more customized product suggestions.

[0549] Step 6:

[0550] The generated product suggestions are sent to the user's terminal, where the user views the suggested products and provides feedback. The server receives the feedback, re-evaluates them, and generates suggestions again if necessary. The input is the user's feedback, and the output is the revised product suggestions.

[0551] Step 7:

[0552] The server performs a process based on this feedback, updating the generating AI model to improve the accuracy of future suggestions. The server constantly learns from new data to improve the performance of product suggestions.

[0553] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0554] The system of the present invention is designed to provide personalized product recommendations to users on an e-commerce platform, and aims to provide a more advanced user experience by incorporating emotion recognition technology.

[0555] Users first use their device to inquire about the products or services they wish to purchase using natural language via a chat box or voice input interface on the e-commerce platform.

[0556] The server receives inquiries from users and uses a natural language processing engine to analyze the content and intent of the requests. Simultaneously, an emotion engine recognizes the user's emotional state from their input, identifying emotions such as joy, surprise, and sadness. This allows the server to understand what emotions are behind the user's requests.

[0557] Based on the analyzed requests and emotional states, the server collects the user's past purchase history and preference data from a database, integrates and analyzes this information, and then applies emotional information to customize the recommendations.

[0558] Furthermore, it utilizes generative artificial intelligence to generate optimal product recommendations. Here, the emotional state identified by the emotion engine is used to determine the priority of product recommendations and to highlight specific product groups. For example, if the user expresses the emotion of "surprise," unique and new products will be prioritized in the recommendations.

[0559] The generated product suggestions are further customized by incorporating multimodal data. If a user uploads images or videos from their device, that data is analyzed, and the suggestions are adjusted to match the user's visual preferences.

[0560] Finally, the server presents the generated suggestions to the user's terminal. The user can view the suggested products and provide additional feedback. The server incorporates this feedback and adjusts the suggestions as needed to provide an experience that recommends the most appropriate products to meet the user's needs. For example, if a user requests "I want a bag for everyday use," and the emotion engine determines that the user is experiencing "enjoyment," then stylishly designed and colorful products will be prioritized in the suggestions.

[0561] The following describes the processing flow.

[0562] Step 1:

[0563] Users access e-commerce platforms using their devices, inputting the products or services they wish to purchase in natural language, and submitting inquiries. Chat windows and voice input are used.

[0564] Step 2:

[0565] The server receives a request written in natural language from the user. This request is then passed to a natural language processing engine for analysis.

[0566] Step 3:

[0567] The server uses a natural language processing engine to analyze the user's request, clarifying its intent and requirements. This analysis identifies key keywords and desired product categories.

[0568] Step 4:

[0569] The server uses an emotion engine to recognize the emotional state contained in the user's natural language request. Emotions such as joy, surprise, and sadness are identified, and the user's mental state is recorded.

[0570] Step 5:

[0571] The server retrieves the user's past purchase history and preference data from the database. This data reflects the user's preferences and past purchasing behavior and can be used to make recommendations.

[0572] Step 6:

[0573] The server integrates historical data collected with emotional information obtained from the emotion engine, and uses generative artificial intelligence to generate optimal product suggestions. Here, the suggested products are adjusted according to the user's emotional state.

[0574] Step 7:

[0575] Users can upload images and videos related to products from their devices. This multimodal data is sent to the server to complement the user's visual preferences.

[0576] Step 8:

[0577] The server analyzes multimodal data to identify the user's visual preferences and then customizes the suggestions accordingly. This makes the product list more closely match the user's specific desires.

[0578] Step 9:

[0579] The server then displays the final product suggestions to the user's terminal. The user can view the suggested products and provide further feedback.

[0580] Step 10:

[0581] The server receives user feedback, and based on that feedback, the AI ​​generates suggestions to re-evaluate and adjust them, continuously presenting the product that best suits the user's needs.

[0582] (Example 2)

[0583] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0584] In modern e-commerce platforms, personalized product recommendations are crucial. However, simply basing recommendations on past purchase history and preferences fails to adequately reflect users' actual needs and emotions, limiting their ability to improve satisfaction. Furthermore, when suggesting appropriate products based on user input, emotional states are often overlooked, making it difficult to provide an optimal purchasing experience.

[0585] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0586] In this invention, the server includes means for receiving natural language input from the user and analyzing its content, means for collecting and analyzing information on the user's purchase history and preferences based on the received input, and means for generating appropriate product suggestions using artificial intelligence. This makes it possible to provide more accurate and personalized product suggestions that also take into account the user's emotional information.

[0587] "User-generated natural language input" refers to users conveying information to an electronic platform using expressions from everyday conversation.

[0588] "Means for analyzing content" refers to a device or software that analyzes received natural language input and processes it to understand its meaning and intent.

[0589] "Purchase history and preference information" refers to data related to products and services that a user has previously acquired, as well as their preferences for those products and services.

[0590] "Generative artificial intelligence" refers to an intelligent program or algorithm that automatically generates optimal product and service suggestions based on large amounts of data.

[0591] "Data in diverse formats" refers to a collection of data that encompasses information obtained in different media formats, such as text, images, audio, and video.

[0592] "Visual preference" refers to a user's particular preferences regarding visual elements such as colors, designs, and styles.

[0593] "Emotional information" refers to information that indicates an emotional state, inferred from user input and interactions.

[0594] The present invention provides a system that generates personalized product suggestions when a user purchases goods or services on an e-commerce platform using a terminal, by having the user input requests in natural language and having the server analyze and process those inputs.

[0595] Users can make product requests via text or voice through their device. If a user enters a request such as "I want new sneakers," the server receives the request and analyzes its content using a natural language processing engine. This analysis utilizes common natural language processing libraries and services (e.g., open-source natural language toolkits or commercial cloud-based solutions).

[0596] As part of its analysis, the server uses emotion recognition technology to recognize the user's emotional state. This emotional state helps determine whether the user is experiencing emotions such as "anticipation" or "excitement." This emotional information is collected from a database along with the user's past purchase history and preference data.

[0597] Based on this information, the server uses a generative AI model to generate product suggestions tailored to the user. Specifically, this AI model leverages the latest, user-friendly, and flexible generative technologies to rank products based on text and scores.

[0598] Furthermore, when a user uploads visual data, such as images or videos, from their device, the server analyzes it and adjusts the suggestions to match the user's visual preferences. For this purpose, image processing libraries and advanced computer vision technologies (e.g., machine learning-based image classification tools) are used.

[0599] Users can provide feedback on the product suggestions presented to them, and the server receives this feedback to improve the suggestions. This feedback loop ensures that the system always provides product suggestions that meet user needs.

[0600] For example, when a user enters the prompt "I want a bag for everyday use," if the emotion engine determines that the user is feeling "excitement," stylish and brightly colored products will be prioritized for suggestion. This allows users to efficiently find the products they are looking for, thereby improving their satisfaction with the purchasing experience.

[0601] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0602] Step 1:

[0603] Users use their devices to send requests for goods and services in natural language through a chat box or voice input interface on the e-commerce platform. The input is received as text or voice data on the device and sent to the server over the network. For example, a user might type or voice the sentence "I want new sneakers." This input becomes the data sent to the server.

[0604] Step 2:

[0605] The server receives natural language input from the user. Using a natural language processing engine, the server analyzes the input data and extracts the requested product category and the user's intent. The input, received as text data, is passed to a language analysis system, where key words and phrases are extracted. As a result, categories such as sneakers and sports shoes are output.

[0606] Step 3:

[0607] The server uses an emotion engine to recognize the user's emotional state from the analyzed input. It analyzes text and voice tone as input data to obtain emotional information such as positive or negative. Specifically, the system determines the emotion "expectation" from the input "I want new sneakers" and outputs emotional information.

[0608] Step 4:

[0609] Based on the intent and sentiment information obtained in the previous step, the server collects and analyzes the user's purchase history and preference data from the database. Based on the acquired past purchase information, data calculations are performed to select highly relevant products. Frequently purchased color palettes and brands are identified from the shopping history and output.

[0610] Step 5:

[0611] The server uses a generative AI model to generate optimal product recommendations. Based on emotional information and purchase history, it outputs a list of suggested products. The generative AI model forms a group of products that it deems optimal for the user, based on past data and algorithms. Specifically, the AI ​​model considers the emotion of "expectation" and generates a list that prioritizes the latest sneaker models.

[0612] Step 6:

[0613] The server analyzes visual information from the user, including images and videos if available, and incorporates that information into the suggested products. Image files uploaded by the user are provided as input, and color and design patterns are extracted using image analysis technology. As a result, a product list matching the user's visual preferences is output and adjusted.

[0614] Step 7:

[0615] The server sends the final generated product suggestions to the user's terminal, where the user views the suggestions and provides feedback. The server analyzes the user's ratings and comments, and adjusts the suggestions if necessary. This feedback is used to improve the system in the next iteration.

[0616] (Application Example 2)

[0617] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0618] On e-commerce platforms, it is not easy for users to find products that suit their individual needs from a wide range of options, which can result in an unsatisfactory purchasing experience. Furthermore, traditional systems lack personalization that takes user emotions into account, resulting in a failure to provide suggestions that match the user's psychological state.

[0619] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0620] In this invention, the server includes means for receiving and analyzing requests in natural language from the user; means for collecting and analyzing the user's past purchase history and preference data based on the received requests; means for generating optimal product suggestions using generative artificial intelligence; means for customizing the suggestion content using multimodal data; means for presenting the generated product suggestions to the user, receiving feedback and responding accordingly; and means for recognizing the user's emotional state and adjusting the product suggestions based on those emotions. This enables personalized product suggestions that respond to the user's emotions.

[0621] "Natural language" refers to the system of words and sentences that humans use on a daily basis, and in this invention, it is a means used to interpret user requests.

[0622] "Means for analysis" refers to a system that has the processing capability to identify, classify, and understand input information based on its meaning and intent, and in this invention, it is used to decipher the user's requests.

[0623] "Purchase history" refers to a record of products and services that a user has purchased in the past. This information is used to identify the user's preferences and provide personalized recommendations.

[0624] "Preference data" refers to data that shows a user's preferences and tendencies, and is used to customize suggestions based on past behavior and choices.

[0625] "Generative artificial intelligence" refers to algorithms and models generated to solve problems according to specific purposes, and in this invention, it is used to create optimal product suggestions.

[0626] "Multimodal data" refers to data in multiple different formats (e.g., images, audio, text) and is used to enrich the proposed content in order to respond to user requests from multiple perspectives.

[0627] "Emotional state" refers to information that represents the user's psychological and emotional tendencies and circumstances, and is used to optimize the suggested content to match the user's emotions.

[0628] "Feedback" refers to user reactions and evaluations, and is information incorporated to improve product offerings.

[0629] The system used to realize this application example includes an e-commerce platform to which users connect using devices such as smartphones and computers. A server performs various processes on this platform.

[0630] First, the user sends their purchase request from their device to the server using natural language. The server receives this input and uses a natural language processing engine, such as the Google Cloud Natural Language API, to analyze the user's request and understand its intent. The information analyzed by the server is integrated with the user's purchase history and preference data. Additionally, sentiment recognition modules such as TensorFlow are used to determine the user's emotional state.

[0631] Next, generative artificial intelligence is used to generate product suggestions that best suit the user's requests and emotions. This process employs a generative AI model such as OpenAI, which determines the suggestions by referring to the user's individual prompt text. For example, a prompt text such as "Female in her 30s, wants a new spring bag, expresses excitement, has a history of purchasing imported goods. Please suggest three recommended products" is supplied to the model, and the generative AI constructs a specific product list.

[0632] Furthermore, the server performs multimodal data analysis to identify visual preferences from images and videos uploaded by the user and adjusts product recommendations accordingly. Image recognition APIs are used in this activity. Finally, the optimized product recommendations are delivered to the user's device. The user can view the presented products and provide feedback based on their preferences. The server uses this feedback to further improve future recommendations and increase user satisfaction.

[0633] This system allows users in e-commerce to experience more appropriate and appealing product suggestions tailored to their emotions and preferences.

[0634] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0635] Step 1:

[0636] A user accesses the e-commerce platform using their device and enters their purchase request in natural language. This input is sent to the server as text data. This input forms the basis for subsequent processing.

[0637] Step 2:

[0638] The server analyzes natural language requests received using the Google Cloud Natural Language API. Through this analysis, it identifies the user's intent and request content, and extracts keywords and categories related to the request. The output is structured data representing the user's request.

[0639] Step 3:

[0640] The server uses TensorFlow to perform emotion recognition from user input. It analyzes the input natural language data to identify specific emotional states such as joy, surprise, and sadness. The output is data representing the user's emotional state.

[0641] Step 4:

[0642] The server collects the user's past purchase history and preference data from the database and analyzes it in combination with the requests and emotional states extracted in the previous step. This process integrates data to gain a deeper understanding of the user's individual needs.

[0643] Step 5:

[0644] The server uses OpenAI's generative AI model to generate product suggestions based on the user's requests and emotions. It provides prompts to the generative AI model, giving specific instructions such as, "Female, in her 30s, wants a new spring bag, expresses excitement. Has a history of purchasing imported goods. Please suggest three recommended items." The output is a list of customized product suggestions.

[0645] Step 6:

[0646] The server uses an image recognition API to analyze images and video data uploaded by users and identify their visual preferences. This analysis allows the server to adjust product recommendations based on visual data. The output is a filtered list of products corresponding to the user's visual preferences.

[0647] Step 7:

[0648] Finally, the server integrates product recommendations based on the analysis and generated results, and transmits a personalized list to the user's device. The user can view this list and provide feedback. This feedback is sent to the server and used to further optimize the recommendations.

[0649] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0650] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0651] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0652] [Fourth Embodiment]

[0653] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0654] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0655] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0656] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0657] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0658] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0659] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0660] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0661] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0662] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0663] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0664] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0665] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0666] The system of this invention is designed to be integrated into an e-commerce platform and to provide personalized product recommendations to users. A specific example of its operation is shown below.

[0667] First, the user accesses an e-commerce site through their device and sends an inquiry in natural language about the product or service they are considering purchasing. This operation is typically performed via a chat box or voice input interface.

[0668] When the server receives a request from a user, it first passes it to a natural language processing engine. Here, the user's inquiry is analyzed to clarify the intent and content of the request. For example, if a user enters "I want running shoes," the server extracts the keywords "running" and "shoes" and focuses on the sports shoes category.

[0669] Next, the server accesses the database to collect the user's past purchase history and preference data. This allows it to identify what products the user has previously purchased, as well as their preferred brands and styles. Based on this information, the generative artificial intelligence selects and suggests products that it believes are best suited to the user.

[0670] Furthermore, users can upload additional information using their devices, such as images and videos related to their purchases. The server analyzes this multimodal data and customizes product recommendations based on the user's visual preferences.

[0671] After product suggestions are generated, the server presents them to the user's terminal. The user has the opportunity to view the suggested product list and provide feedback. For example, if the user requests that "the design should be more colorful," the server uses the generating AI to re-evaluate the suggestions and present new, more appropriate products.

[0672] In this way, the system aims to provide product suggestions that more accurately meet user needs and improve the purchasing experience. Through this series of processes, the interaction between users and the e-commerce platform becomes more efficient and satisfying.

[0673] The following describes the processing flow.

[0674] Step 1:

[0675] Users access e-commerce platforms using their devices and enter requests for desired products or services in natural language. This is typically done through the site's chat window.

[0676] Step 2:

[0677] The server receives a request from the user. This information is sent to a natural language processing engine, which analyzes the request and clarifies its content and intent.

[0678] Step 3:

[0679] Based on the analyzed information, the server retrieves the user's past purchase history and preference data from a database. This enables product recommendations that take into account the user's preferences and past behavior.

[0680] Step 4:

[0681] The server uses artificial intelligence to integrate user requests with historical data to generate an optimal product list. This list includes recommended products and similar items.

[0682] Step 5:

[0683] Users upload images and videos related to products using their devices. This multimodal data is sent to the server as additional preference information.

[0684] Step 6:

[0685] The server analyzes multimodal data to identify the user's visual preferences. Based on this information, it customizes the suggested product list to create more specific product recommendations.

[0686] Step 7:

[0687] The server then displays the final product suggestions generated by the server to the user's terminal. The user can browse these products and provide further requests or feedback.

[0688] Step 8:

[0689] The server receives user feedback, uses AI to re-evaluate the suggested products based on that feedback, and then presents a further refined product list to provide suggestions that best meet the user's needs.

[0690] (Example 1)

[0691] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] Modern e-commerce platforms offer generic and uniform product suggestions to users, failing to adequately reflect each user's individual preferences and past purchase history. Therefore, there is a need to develop systems that can satisfy users' purchasing desires and assist them in making optimal product choices.

[0693] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0694] In this invention, the server includes means for receiving and analyzing requests from users in natural language, means for collecting and analyzing the user's past purchase history and preference information based on the received requests, and means for generating optimal product suggestions using generative artificial intelligence. This enables personalized product suggestions for each user and improves the purchasing experience.

[0695] "Requests made in natural language" refer to sentences or phrases that users use to communicate their intentions in text or voice, without being constrained by any specific format.

[0696] "Purchase history" refers to detailed information and records of products that a user has purchased in the past.

[0697] "Preference information" refers to data that shows a user's preferences and tendencies, and represents personal judgment criteria that influence product selection.

[0698] "Generative artificial intelligence" refers to an algorithm or system that has the ability to generate product suggestions tailored to the user using machine learning techniques.

[0699] "Diverse media data" refers to data in different formats, including visual information such as images and videos, and auditory information such as audio.

[0700] "Product recommendations" refer to a list of products and services recommended based on the user's needs and preferences.

[0701] "Opinions" refer to the thoughts, requests, and wishes that users express regarding the proposed product.

[0702] The embodiments for carrying out the present invention will now be described. This system implements advanced processing for providing personalized product recommendations to users. Its specific operation is shown below.

[0703] First, users use their devices to access e-commerce platforms and then submit inquiries in natural language about desired products or services they are interested in. These natural language inquiries are typically made through chat boxes or voice input interfaces.

[0704] Next, the server receives the user's request and first analyzes it using a natural language processing (NLP) engine. The analysis clarifies the user's intent and request. Based on this information, if the user says "I want running shoes," the server extracts the keywords "running" and "shoes" and identifies the product category to suggest.

[0705] Next, the server accesses an internal database to collect data based on the user's past purchase history and preferences. This allows the server to identify products the user has previously purchased, as well as their preferred brands and styles. This data serves as crucial foundational information for suggesting the most suitable products to the user.

[0706] Subsequently, the server uses a generative AI model to generate optimal product recommendations for the user based on this analytical data. The generative AI model is based on machine learning algorithms and lists the most suitable products while taking into account the user's past preferences.

[0707] Furthermore, users can upload images and videos related to their purchases through their devices. The server analyzes this multimodal data and further customizes product recommendations based on the user's visual preferences. In this process, preferences for color and design are particularly important factors.

[0708] Finally, the server displays the generated product suggestions to the user's device. The user has the opportunity to view the presented product list and provide feedback. This feedback will be used to further improve the suggestions.

[0709] For example, if a user enters "I want a tent I can use for camping," the server will refer to the user's past purchase history of outdoor equipment based on the keywords "camping" and "tent," and suggest appropriate products. An example of a prompt to the generating AI model is, "Please suggest new products related to the camping equipment the user has purchased in the past."

[0710] This will allow the system to accurately meet user needs and improve the purchasing experience.

[0711] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0712] Step 1:

[0713] Users use their devices to access e-commerce platforms and submit inquiries in natural language about products and services they are considering purchasing. This input can be in text or voice format and is sent from the device to the server. For example, a user might ask, "I want running shoes."

[0714] Step 2:

[0715] The server receives requests from users in natural language and passes them to a natural language processing (NLP) engine for analysis. The input text is extracted through analysis into keywords such as "running" and "shoes." This output clarifies the user's intent and request.

[0716] Step 3:

[0717] Based on user requests, the server accesses a database within the system to collect the user's past purchase history and preference information. Database queries are sent as input, the data is processed, and the output includes data on the user's preferences and past purchases. Specifically, this includes information on previously purchased sports shoes and brands.

[0718] Step 4:

[0719] The server utilizes a generative AI model to generate optimal product suggestions based on the user's past data and received requests. The AI ​​model uses machine learning algorithms to analyze a large amount of user data and suggest new products. The input for this step is past preference data and product category information, and the output is a personalized list of products.

[0720] Step 5:

[0721] Users can upload images and videos related to their purchases using their devices. The server receives these and analyzes them as multimodal data. The visual data used as input is analyzed to identify the user's visual preferences, which are then reflected in the most suitable recommendations. For example, if a user prefers colorful designs, that information is taken into account in the product recommendations.

[0722] Step 6:

[0723] The server presents customized product suggestions to the user's terminal. It uses a generated product list as input and employs prompts to present relevant information to the user. As output, the user receives the product list and can review the details. The user can then provide feedback and return data to the server to further improve the accuracy of the suggestions.

[0724] (Application Example 1)

[0725] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0726] Providing products that quickly and appropriately meet the diverse needs of users in online shopping is challenging. In this situation, simply suggesting products based on a user's past purchase history and basic information is insufficient; more advanced analysis and personalization are required. Furthermore, it is essential to flexibly update suggestions using user feedback. The ability to accurately grasp user needs using natural language input and multimodal data is also crucial. Solving these challenges is essential.

[0727] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0728] In this invention, the server includes means for receiving and analyzing requests in natural language from users, means for collecting and analyzing the user's past purchase history and preference data, means for generating optimal product suggestions using generative artificial intelligence, and means for providing customized product suggestions using machine learning models. This enables rapid and highly accurate product suggestions to meet the diverse needs of users.

[0729] "Requests in natural language" refer to requests or questions that users input using everyday language.

[0730] "Means of analysis" refers to a system or program that has the function of breaking down received information and performing a process to extract its meaning and intent.

[0731] "Purchase history" refers to a record of products and services that a user has purchased in the past.

[0732] "Preference data" refers to information about preferences and trends collected based on a user's past preferences, interests, and usage history.

[0733] "Generative artificial intelligence" refers to machine learning techniques and algorithms that dynamically generate optimal suggestions and answers based on user requests and preferences.

[0734] "Multimodal data" refers to information that combines multiple types of data, such as audio, images, and text.

[0735] "Means of customization" refers to methods and technologies for adjusting and optimizing output according to the individual user's needs and preferences.

[0736] "Feedback" refers to the information provided by users regarding their user experience and suggestions, and is used to improve the system's performance.

[0737] "Natural language processing capabilities" refer to the technology that enables computers to understand and process human language.

[0738] A "data storage structure" refers to a system of databases and file systems designed to efficiently store and utilize information.

[0739] A "machine learning model" refers to an algorithm that learns from large amounts of data and recognizes patterns to make future predictions and classifications.

[0740] In order to implement this invention, it is mainly necessary for a server and a user terminal to work together. The server has advanced computing resources and is responsible for processing user requests and generating appropriate product suggestions. User terminals refer to various devices such as smartphones and computers, and are means for users to interface with the system.

[0741] When a server receives a request from a user in natural language, it first uses a natural language processing engine to analyze the request. Typically, machine learning-based natural language processing technologies, such as OpenAI's API, are used for this process. Based on the analysis results, the server identifies the user's intent and emotions regarding the request.

[0742] Next, the server retrieves the user's past purchase history and preference data from the database. This data is managed by a database management system such as SQLite. Once the information is collected, the server uses machine learning models, such as TensorFlow or PyTorch, to create generative AI models and make optimal product recommendations.

[0743] Furthermore, if multimodal data such as images and videos are available from the user, this data is used to further customize the suggestions. At this time, computer vision technology and deep learning are used to analyze the user's visual preferences and generate new feedback.

[0744] The generated product suggestions are sent to the user's device, where the user evaluates them. If feedback is submitted, for example, if the user suggests "a more colorful design would be better," the server re-evaluates the suggestion and updates it. In this way, the system can continuously provide the best suggestions that meet the user's needs.

[0745] As a concrete example, the following is an example of a prompt message that analyzes the request, "I want lightweight, waterproof running shoes."

[0746] Example of a prompt:

[0747] "Analyze the user's query: 'I want lightweight, waterproof running shoes,' and identify the requested product category and characteristics."

[0748] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0749] Step 1:

[0750] The user uses a device to input a request using natural language, either by voice or text. This request clearly indicates the user's purchasing interest and is specific, such as "I want lightweight, waterproof running shoes." The device then sends this request to the server.

[0751] Step 2:

[0752] The server receives requests written in natural language and analyzes them using a natural language processing engine. This analysis breaks down the user's request into keywords, extracting information such as "lightweight," "waterproof," and "running shoes." The input is the user's request, and the output is a set of analyzed keywords.

[0753] Step 3:

[0754] The server then retrieves and aggregates user purchase history and preference data from the database. At this stage, a database management system such as SQLite is used. The input is the user's ID, and the output is the user's past purchase history and preference data. This information is used to process the data and analyze user interests and preferences.

[0755] Step 4:

[0756] The server uses a machine learning model to generate optimal product suggestions based on the analysis results and user preference data. Here, a generative AI model is used to generate a list of products best suited to satisfy the user's requirements. The input is the output data from steps 2 and 3, and the output is a list of product suggestions.

[0757] Step 5:

[0758] If the user submits additional images or videos, this data is analyzed and multimodal data is used to further customize product suggestions. Image processing techniques are used to analyze the user's visual preferences. The input is image or video data from the user, and the output is a list of more customized product suggestions.

[0759] Step 6:

[0760] The generated product suggestions are sent to the user's terminal, where the user views the suggested products and provides feedback. The server receives the feedback, re-evaluates them, and generates suggestions again if necessary. The input is the user's feedback, and the output is the revised product suggestions.

[0761] Step 7:

[0762] The server performs a process based on this feedback, updating the generating AI model to improve the accuracy of future suggestions. The server constantly learns from new data to improve the performance of product suggestions.

[0763] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0764] The system of the present invention is designed to provide personalized product recommendations to users on an e-commerce platform, and aims to provide a more advanced user experience by incorporating emotion recognition technology.

[0765] Users first use their device to inquire about the products or services they wish to purchase using natural language via a chat box or voice input interface on the e-commerce platform.

[0766] The server receives inquiries from users and uses a natural language processing engine to analyze the content and intent of the requests. Simultaneously, an emotion engine recognizes the user's emotional state from their input, identifying emotions such as joy, surprise, and sadness. This allows the server to understand what emotions are behind the user's requests.

[0767] Based on the analyzed requests and emotional states, the server collects the user's past purchase history and preference data from a database, integrates and analyzes this information, and then applies emotional information to customize the recommendations.

[0768] Furthermore, it utilizes generative artificial intelligence to generate optimal product recommendations. Here, the emotional state identified by the emotion engine is used to determine the priority of product recommendations and to highlight specific product groups. For example, if the user expresses the emotion of "surprise," unique and new products will be prioritized in the recommendations.

[0769] The generated product suggestions are further customized by incorporating multimodal data. If a user uploads images or videos from their device, that data is analyzed, and the suggestions are adjusted to match the user's visual preferences.

[0770] Finally, the server presents the generated suggestions to the user's terminal. The user can view the suggested products and provide additional feedback. The server incorporates this feedback and adjusts the suggestions as needed to provide an experience that recommends the most appropriate products to meet the user's needs. For example, if a user requests "I want a bag for everyday use," and the emotion engine determines that the user is experiencing "enjoyment," then stylishly designed and colorful products will be prioritized in the suggestions.

[0771] The following describes the processing flow.

[0772] Step 1:

[0773] Users access e-commerce platforms using their devices, inputting the products or services they wish to purchase in natural language, and submitting inquiries. Chat windows and voice input are used.

[0774] Step 2:

[0775] The server receives a request written in natural language from the user. This request is then passed to a natural language processing engine for analysis.

[0776] Step 3:

[0777] The server uses a natural language processing engine to analyze the user's request, clarifying its intent and requirements. This analysis identifies key keywords and desired product categories.

[0778] Step 4:

[0779] The server uses an emotion engine to recognize the emotional state contained in the user's natural language request. Emotions such as joy, surprise, and sadness are identified, and the user's mental state is recorded.

[0780] Step 5:

[0781] The server retrieves the user's past purchase history and preference data from the database. This data reflects the user's preferences and past purchasing behavior and can be used to make recommendations.

[0782] Step 6:

[0783] The server integrates historical data collected with emotional information obtained from the emotion engine, and uses generative artificial intelligence to generate optimal product suggestions. Here, the suggested products are adjusted according to the user's emotional state.

[0784] Step 7:

[0785] Users can upload images and videos related to products from their devices. This multimodal data is sent to the server to complement the user's visual preferences.

[0786] Step 8:

[0787] The server analyzes multimodal data to identify the user's visual preferences and then customizes the suggestions accordingly. This makes the product list more closely match the user's specific desires.

[0788] Step 9:

[0789] The server then displays the final product suggestions to the user's terminal. The user can view the suggested products and provide further feedback.

[0790] Step 10:

[0791] The server receives user feedback, and based on that feedback, the AI ​​generates suggestions to re-evaluate and adjust them, continuously presenting the product that best suits the user's needs.

[0792] (Example 2)

[0793] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0794] In modern e-commerce platforms, personalized product recommendations are crucial. However, simply basing recommendations on past purchase history and preferences fails to adequately reflect users' actual needs and emotions, limiting their ability to improve satisfaction. Furthermore, when suggesting appropriate products based on user input, emotional states are often overlooked, making it difficult to provide an optimal purchasing experience.

[0795] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0796] In this invention, the server includes means for receiving natural language input from the user and analyzing its content, means for collecting and analyzing information on the user's purchase history and preferences based on the received input, and means for generating appropriate product suggestions using artificial intelligence. This makes it possible to provide more accurate and personalized product suggestions that also take into account the user's emotional information.

[0797] "User-generated natural language input" refers to users conveying information to an electronic platform using expressions from everyday conversation.

[0798] "Means for analyzing content" refers to a device or software that analyzes received natural language input and processes it to understand its meaning and intent.

[0799] "Purchase history and preference information" refers to data related to products and services that a user has previously acquired, as well as their preferences for those products and services.

[0800] "Generative artificial intelligence" refers to an intelligent program or algorithm that automatically generates optimal product and service suggestions based on large amounts of data.

[0801] "Data in diverse formats" refers to a collection of data that encompasses information obtained in different media formats, such as text, images, audio, and video.

[0802] "Visual preference" refers to a user's particular preferences regarding visual elements such as colors, designs, and styles.

[0803] "Emotional information" refers to information that indicates an emotional state, inferred from user input and interactions.

[0804] The present invention provides a system that generates personalized product suggestions when a user purchases goods or services on an e-commerce platform using a terminal, by having the user input requests in natural language and having the server analyze and process those inputs.

[0805] Users can make product requests via text or voice through their device. If a user enters a request such as "I want new sneakers," the server receives the request and analyzes its content using a natural language processing engine. This analysis utilizes common natural language processing libraries and services (e.g., open-source natural language toolkits or commercial cloud-based solutions).

[0806] As part of its analysis, the server uses emotion recognition technology to recognize the user's emotional state. This emotional state helps determine whether the user is experiencing emotions such as "anticipation" or "excitement." This emotional information is collected from a database along with the user's past purchase history and preference data.

[0807] Based on this information, the server uses a generative AI model to generate product suggestions tailored to the user. Specifically, this AI model leverages the latest, user-friendly, and flexible generative technologies to rank products based on text and scores.

[0808] Furthermore, when a user uploads visual data, such as images or videos, from their device, the server analyzes it and adjusts the suggestions to match the user's visual preferences. For this purpose, image processing libraries and advanced computer vision technologies (e.g., machine learning-based image classification tools) are used.

[0809] Users can provide feedback on the product suggestions presented to them, and the server receives this feedback to improve the suggestions. This feedback loop ensures that the system always provides product suggestions that meet user needs.

[0810] For example, when a user enters the prompt "I want a bag for everyday use," if the emotion engine determines that the user is feeling "excitement," stylish and brightly colored products will be prioritized for suggestion. This allows users to efficiently find the products they are looking for, thereby improving their satisfaction with the purchasing experience.

[0811] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0812] Step 1:

[0813] Users use their devices to send requests for goods and services in natural language through a chat box or voice input interface on the e-commerce platform. The input is received as text or voice data on the device and sent to the server over the network. For example, a user might type or voice the sentence "I want new sneakers." This input becomes the data sent to the server.

[0814] Step 2:

[0815] The server receives natural language input from the user. Using a natural language processing engine, the server analyzes the input data and extracts the requested product category and the user's intent. The input, received as text data, is passed to a language analysis system, where key words and phrases are extracted. As a result, categories such as sneakers and sports shoes are output.

[0816] Step 3:

[0817] The server uses an emotion engine to recognize the user's emotional state from the analyzed input. It analyzes text and voice tone as input data to obtain emotional information such as positive or negative. Specifically, the system determines the emotion "expectation" from the input "I want new sneakers" and outputs emotional information.

[0818] Step 4:

[0819] Based on the intent and sentiment information obtained in the previous step, the server collects and analyzes the user's purchase history and preference data from the database. Based on the acquired past purchase information, data calculations are performed to select highly relevant products. Frequently purchased color palettes and brands are identified from the shopping history and output.

[0820] Step 5:

[0821] The server uses a generative AI model to generate optimal product recommendations. Based on emotional information and purchase history, it outputs a list of suggested products. The generative AI model forms a group of products that it deems optimal for the user, based on past data and algorithms. Specifically, the AI ​​model considers the emotion of "expectation" and generates a list that prioritizes the latest sneaker models.

[0822] Step 6:

[0823] The server analyzes visual information from the user, including images and videos if available, and incorporates that information into the suggested products. Image files uploaded by the user are provided as input, and color and design patterns are extracted using image analysis technology. As a result, a product list matching the user's visual preferences is output and adjusted.

[0824] Step 7:

[0825] The server sends the final generated product suggestions to the user's terminal, where the user views the suggestions and provides feedback. The server analyzes the user's ratings and comments, and adjusts the suggestions if necessary. This feedback is used to improve the system in the next iteration.

[0826] (Application Example 2)

[0827] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0828] On e-commerce platforms, it is not easy for users to find products that suit their individual needs from a wide range of options, which can result in an unsatisfactory purchasing experience. Furthermore, traditional systems lack personalization that takes user emotions into account, resulting in a failure to provide suggestions that match the user's psychological state.

[0829] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0830] In this invention, the server includes means for receiving and analyzing requests in natural language from the user; means for collecting and analyzing the user's past purchase history and preference data based on the received requests; means for generating optimal product suggestions using generative artificial intelligence; means for customizing the suggestion content using multimodal data; means for presenting the generated product suggestions to the user, receiving feedback and responding accordingly; and means for recognizing the user's emotional state and adjusting the product suggestions based on those emotions. This enables personalized product suggestions that respond to the user's emotions.

[0831] "Natural language" refers to the system of words and sentences that humans use on a daily basis, and in this invention, it is a means used to interpret user requests.

[0832] "Means for analysis" refers to a system that has the processing capability to identify, classify, and understand input information based on its meaning and intent, and in this invention, it is used to decipher the user's requests.

[0833] "Purchase history" refers to a record of products and services that a user has purchased in the past. This information is used to identify the user's preferences and provide personalized recommendations.

[0834] "Preference data" refers to data that shows a user's preferences and tendencies, and is used to customize suggestions based on past behavior and choices.

[0835] "Generative artificial intelligence" refers to algorithms and models generated to solve problems according to specific purposes, and in this invention, it is used to create optimal product suggestions.

[0836] "Multimodal data" refers to data in multiple different formats (e.g., images, audio, text) and is used to enrich the proposed content in order to respond to user requests from multiple perspectives.

[0837] "Emotional state" refers to information that represents the user's psychological and emotional tendencies and circumstances, and is used to optimize the suggested content to match the user's emotions.

[0838] "Feedback" refers to user reactions and evaluations, and is information incorporated to improve product offerings.

[0839] The system used to realize this application example includes an e-commerce platform to which users connect using devices such as smartphones and computers. A server performs various processes on this platform.

[0840] First, the user sends their purchase request from their device to the server using natural language. The server receives this input and uses a natural language processing engine, such as the Google Cloud Natural Language API, to analyze the user's request and understand its intent. The information analyzed by the server is integrated with the user's purchase history and preference data. Additionally, sentiment recognition modules such as TensorFlow are used to determine the user's emotional state.

[0841] Next, generative artificial intelligence is used to generate product suggestions that best suit the user's requests and emotions. This process employs a generative AI model such as OpenAI, which determines the suggestions by referring to the user's individual prompt text. For example, a prompt text such as "Female in her 30s, wants a new spring bag, expresses excitement, has a history of purchasing imported goods. Please suggest three recommended products" is supplied to the model, and the generative AI constructs a specific product list.

[0842] Furthermore, the server performs multimodal data analysis to identify visual preferences from images and videos uploaded by the user and adjusts product recommendations accordingly. Image recognition APIs are used in this activity. Finally, the optimized product recommendations are delivered to the user's device. The user can view the presented products and provide feedback based on their preferences. The server uses this feedback to further improve future recommendations and increase user satisfaction.

[0843] This system allows users in e-commerce to experience more appropriate and appealing product suggestions tailored to their emotions and preferences.

[0844] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0845] Step 1:

[0846] A user accesses the e-commerce platform using their device and enters their purchase request in natural language. This input is sent to the server as text data. This input forms the basis for subsequent processing.

[0847] Step 2:

[0848] The server analyzes natural language requests received using the Google Cloud Natural Language API. Through this analysis, it identifies the user's intent and request content, and extracts keywords and categories related to the request. The output is structured data representing the user's request.

[0849] Step 3:

[0850] The server uses TensorFlow to perform emotion recognition from user input. It analyzes the input natural language data to identify specific emotional states such as joy, surprise, and sadness. The output is data representing the user's emotional state.

[0851] Step 4:

[0852] The server collects the user's past purchase history and preference data from the database and analyzes it in combination with the requests and emotional states extracted in the previous step. This process integrates data to gain a deeper understanding of the user's individual needs.

[0853] Step 5:

[0854] The server uses OpenAI's generative AI model to generate product suggestions based on the user's requests and emotions. It provides prompts to the generative AI model, giving specific instructions such as, "Female, in her 30s, wants a new spring bag, expresses excitement. Has a history of purchasing imported goods. Please suggest three recommended items." The output is a list of customized product suggestions.

[0855] Step 6:

[0856] The server uses an image recognition API to analyze images and video data uploaded by users and identify their visual preferences. This analysis allows the server to adjust product recommendations based on visual data. The output is a filtered list of products corresponding to the user's visual preferences.

[0857] Step 7:

[0858] Finally, the server integrates product recommendations based on the analysis and generated results, and transmits a personalized list to the user's device. The user can view this list and provide feedback. This feedback is sent to the server and used to further optimize the recommendations.

[0859] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0860] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0861] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0862] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0863] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0864] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0865] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0866] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0867] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0868] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0869] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0870] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0871] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0872] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0873] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0874] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0875] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0876] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0877] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0878] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0879] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0880] The following is further disclosed regarding the embodiments described above.

[0881] (Claim 1)

[0882] A means of receiving and analyzing requests in natural language from users,

[0883] A means for collecting and analyzing a user's past purchase history and preference data based on a received request,

[0884] A method for generating optimal product suggestions using generative artificial intelligence,

[0885] A means of customizing the proposed content using multimodal data,

[0886] A means of presenting generated product suggestions to users, receiving feedback, and responding accordingly,

[0887] A system that includes this.

[0888] (Claim 2)

[0889] The system according to claim 1, further comprising means for analyzing the user's intent and emotions using a natural language processing engine.

[0890] (Claim 3)

[0891] The system according to claim 1, further comprising means for using image or video data from the user to identify the user's visual preferences.

[0892] "Example 1"

[0893] (Claim 1)

[0894] A means of receiving and analyzing requests in natural language from users,

[0895] A means for collecting and analyzing a user's past purchase history and preference information based on a received request,

[0896] A means of generating optimal product suggestions using generative artificial intelligence,

[0897] A means of customizing proposals using diverse media data,

[0898] A means of presenting generated item proposals to users, receiving their feedback, and responding accordingly,

[0899] A system that includes this.

[0900] (Claim 2)

[0901] The system according to claim 1, further comprising means for analyzing the user's intent and emotions using a natural language processing engine.

[0902] (Claim 3)

[0903] The system according to claim 1, further comprising means for using image or video data from the user to identify the user's visual preferences.

[0904] "Application Example 1"

[0905] (Claim 1)

[0906] A means of receiving and analyzing requests in natural language from users,

[0907] A means for collecting and analyzing a user's past purchase history and preference data based on a received request,

[0908] A method for generating optimal product suggestions using generative artificial intelligence,

[0909] A means of customizing the proposed content using multimodal data,

[0910] A means of presenting generated product suggestions to users, receiving feedback, and responding accordingly,

[0911] A means of setting up natural language processing functions to analyze user input,

[0912] A means of obtaining user preference information by utilizing a data storage structure,

[0913] A means of providing customized product suggestions using machine learning models,

[0914] A system that includes this.

[0915] (Claim 2)

[0916] The system according to claim 1, further comprising means for analyzing the user's intent and emotions using a natural language processing engine.

[0917] (Claim 3)

[0918] The system according to claim 1, further comprising means for using image or video data from the user to identify the user's visual preferences.

[0919] "Example 2 of combining an emotion engine"

[0920] (Claim 1)

[0921] A means for receiving natural language input from a user and analyzing its content,

[0922] A means for collecting and analyzing information about the user's purchase history and preferences based on the received input,

[0923] A means of creating appropriate product suggestions using the generated artificial intelligence,

[0924] A means of adapting the proposed content using diverse data formats,

[0925] A means of presenting the created product proposals to users, collecting their opinions, and responding to them,

[0926] A means of using emotional information to determine the priority of product suggestions and to emphasize specific product groups,

[0927] A system that includes this.

[0928] (Claim 2)

[0929] The system according to claim 1, further comprising means for analyzing the user's intent and emotional state using natural language processing.

[0930] (Claim 3)

[0931] The system according to claim 1, further comprising means for identifying the user's visual preferences and adjusting the suggested content using visual data from the user.

[0932] "Application example 2 when combining with an emotional engine"

[0933] (Claim 1)

[0934] A means of receiving and analyzing requests in natural language from users,

[0935] A means for collecting and analyzing a user's past purchase history and preference data based on a received request,

[0936] A method for generating optimal product suggestions using generative artificial intelligence,

[0937] A means of customizing the proposed content using multimodal data,

[0938] A means of presenting generated product suggestions to users, receiving feedback, and responding accordingly,

[0939] A means of recognizing the user's emotional state and adjusting product suggestions based on that emotion,

[0940] A system that includes this.

[0941] (Claim 2)

[0942] The system according to claim 1, further comprising means for analyzing the user's intent and emotions using a natural language processing engine.

[0943] (Claim 3)

[0944] The system according to claim 1, further comprising means for using image or video data from the user to identify the user's visual preferences. [Explanation of symbols]

[0945] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving and analyzing requests in natural language from users, A means for collecting and analyzing a user's past purchase history and preference data based on a received request, A method for generating optimal product suggestions using generative artificial intelligence, A means of customizing the proposed content using multimodal data, A means of presenting generated product suggestions to users, receiving feedback, and responding accordingly, A means of setting up natural language processing functions to analyze user input, A means of obtaining user preference information by utilizing a data storage structure, A means of providing customized product suggestions using machine learning models, A system that includes this.

2. The system according to claim 1, further comprising means for analyzing the user's intent and emotions using a natural language processing engine.

3. The system according to claim 1, further comprising means for identifying the user's visual preferences using image or video data from the user.

Images