system

Abstract

We provide the system. [Solution] A means for the user to input existing information, A means by which the server receives existing information sent by the user and retrieves related information from the database, The server preprocesses the collected information and extracts features, A means by which a server inputs pre-processed data into a generative artificial intelligence system to generate an optimal plan, A means by which the server converts the generated plan into a predetermined format and sends it to the terminal, A means by which users provide feedback and the server updates the plan based on that feedback, A system that includes this.

Description

Technical Field

[0005] ,

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003] <000001​​​​​​​​​​​​​​​​​​​To solve the above problems, the present invention provides the following means: A means is provided for the user to input existing information, thereby collecting user profile data and request data. The server receives the existing information sent by the user and further retrieves related information from a database to collect detailed existing information of the user. The server preprocesses the collected information and extracts features to organize and analyze the information. Next, the server inputs the preprocessed data into a generative artificial intelligence and generates a customized plan suitable for the user using a means for generating an optimal plan. The server converts the generated plan into a predetermined format and presents the plan in a format suitable for the user terminal using a means for sending it to the terminal. Furthermore, the user provides feedback, and the server updates the plan based on that feedback, dynamically adjusting the plan based on the user's needs. This improves user satisfaction and realizes a flexible system that can respond to individual needs.

[0006] A "user" refers to an individual or legal entity that uses the system, enters their own information, and receives services or plans.

[0007] "Existing information" refers to information that users provide to the system, including past history, profile data, preferences, and other data.

[0008] A "server" refers to a computer system that receives, processes, and analyzes data sent by users.

[0009] A "database" refers to a storage system that systematically stores existing user information and related data, making it accessible as needed.

[0010] "Preprocessing" refers to processes such as data cleaning, normalization, and imputation that are performed to transform collected data into a format that is easy to analyze.

[0011] "Features" refer to important attributes and metrics extracted from data that machine learning models use when performing analysis.

[0012] "Generative artificial intelligence" refers to machine learning algorithms that automatically generate the optimal plan based on input data.

[0013] A "plan" refers to a proposal or schedule generated based on user needs, and includes specific action plans and service combinations.

[0014] A "device" refers to a device used by a user for interaction and viewing information, and includes smartphones, tablets, and computers.

[0015] "Feedback" refers to the act of a user sending opinions or additional requests to the system regarding the plan provided. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]Shows an emotion map to which a plurality of emotions are mapped. [Figure 10] Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the language used in the following description will be described.

[0019] <​​​​In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention relates to a system that automatically generates an optimal plan using generative artificial intelligence based on the user's existing information and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[0038] In this system, the user, terminal, and server work together to generate a plan tailored to the user's needs. The following processes are performed in order to implement the invention.

[0039] Program processing

[0040] 1. User Input Phase

[0041] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input, "I want an exercise and meal plan to lose 5kg."

[0042] 2. Data Collection Phase

[0043] The terminal sends the user's input information to the server.

[0044] The server receives information sent by the user and retrieves related information from the database (e.g., past health data, purchase history, etc.).

[0045] 3. Data preprocessing and feature extraction phase

[0046] The server preprocesses the collected information. Preprocessing includes data cleaning, normalization, and removal of outliers.

[0047] The server extracts features from the pre-processed data by running it through a machine learning model. For example, these features might include "the user's weight fluctuation patterns and exercise preferences."

[0048] 4. Plan Generation Phase

[0049] The server inputs the extracted features into a generative artificial intelligence system and instructs it to generate the optimal plan for the user.

[0050] Generative artificial intelligence analyzes the input data and generates the optimal plan for the user. For example, it might recommend a plan that includes jogging three times a week and a low-fat diet.

[0051] 5. Plan Presentation Phase

[0052] The server converts the generated plan into a predetermined format and sends it to the terminal.

[0053] The device will display the plan to the user. For example, the user could view the detailed plan via a notification on their smartphone.

[0054] 6. Dialogue and Feedback Phase

[0055] Users ask questions and provide feedback about the generated plan. For example, they might say, "Jogging is difficult. Are there any other exercises?"

[0056] The device sends this feedback to the server.

[0057] The server analyzes the feedback and regenerates the plan as needed. Generative artificial intelligence generates a new plan based on the feedback, and the server sends it to the terminal. For example, it might suggest "cycling three times a week instead."

[0058] Specific example

[0059] Let's explain using a specific example of a user requesting a health plan.

[0060] 1. User input phase:

[0061] The user uses their device to input, "I want an exercise and meal plan to lose 5kg."

[0062] 2. Data Collection Phase:

[0063] The terminal sends this input to the server.

[0064] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database.

[0065] 3. Data preprocessing and feature extraction phase:

[0066] The server removes outliers from the collected data and normalizes the data.

[0067] The server uses machine learning models to extract features from the user's health data, such as weight fluctuation patterns and preferred exercise types.

[0068] 4. Plan generation phase:

[0069] The server inputs these features into a generative artificial intelligence system and instructs it to generate a plan that suits the user's needs.

[0070] The generative artificial intelligence generates a plan recommending "jogging three times a week and a low-fat diet."

[0071] 5. Plan presentation phase:

[0072] The server sends the generated plan to the terminal.

[0073] The device notifies the user and displays details within the app.

[0074] 6. Dialogue and Feedback Phase:

[0075] Users send feedback through their devices, saying things like, "Jogging is difficult. Are there any other exercises?"

[0076] The device sends this feedback to the server.

[0077] The server receives feedback and instructs the generative artificial intelligence to generate new movement options.

[0078] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[0079] The server sends the updated plan to the terminal and presents it to the user again.

[0080] As described above, the present invention allows users to receive plans based on their needs quickly and accurately. Furthermore, the plan can be dynamically adjusted through feedback, achieving high flexibility and user satisfaction.

[0081] The following describes the processing flow.

[0082] Step 1:

[0083] The user enters their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "An exercise and meal plan to lose 5kg."

[0084] Step 2:

[0085] The terminal receives information entered by the user and sends that information to the server. This process takes place in real time.

[0086] Step 3:

[0087] After receiving user input, the server retrieves relevant detailed information from the database. For example, it collects data such as past weight fluctuations, dietary records, and exercise history.

[0088] Step 4:

[0089] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization. At this stage, the data is transformed into a consistent format.

[0090] Step 5:

[0091] The server extracts features using pre-processed data. Machine learning models are used to identify important attributes and metrics from the data. For example, it extracts user exercise patterns, dietary habits, and allergy information.

[0092] Step 6:

[0093] The server inputs the extracted features into the generative artificial intelligence and instructs it to generate a plan that suits the user's needs. The generative AI analyzes the input data and automatically generates the optimal plan for the user.

[0094] Step 7:

[0095] Generative artificial intelligence generates specific exercise and meal plans based on the analysis results. For example, it might suggest a plan for jogging three times a week and a low-fat diet.

[0096] Step 8:

[0097] The server receives the generated plan and converts it to a predetermined format before sending it to the user's terminal. This makes the plan easier for the user to understand.

[0098] Step 9:

[0099] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[0100] Step 10:

[0101] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[0102] Step 11:

[0103] The device sends user feedback to the server in real time.

[0104] Step 12:

[0105] The server receives user feedback and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[0106] Step 13:

[0107] Generative artificial intelligence incorporates user feedback to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[0108] Step 14:

[0109] The server receives the updated plan and sends it back to the device.

[0110] Step 15:

[0111] The device will display the plan to the user again and provide updated information.

[0112] Through these steps, the system responds quickly and accurately to user needs, generating and adjusting plans dynamically. This allows users to continuously receive individually customized plans.

[0113] (Example 1)

[0114] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0115] In modern society, health management and the provision of personalized plans are crucial issues. However, existing methods struggle to effectively handle large amounts of data and quickly generate personalized plans. Furthermore, they lack the functionality to dynamically adjust plans based on user feedback. This can lead to users not receiving the most suitable plan, resulting in decreased satisfaction.

[0116] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0117] In this invention, the server includes means for the user to input existing information, means for a terminal to receive existing information transmitted from the user and transfer it to the server, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the plan generated by the generative artificial intelligence into a predetermined format and transmit it to the terminal, means for the terminal to notify and display the plan to the user, means for the user to provide feedback and for the terminal to send the feedback to the server, and means for the server to update the plan based on the feedback. As a result, the user can obtain a personalized plan quickly and accurately, and since the plan is dynamically adjusted based on the feedback, user satisfaction can be increased.

[0118] A "user" refers to an individual who uses the system to request health management and plan generation.

[0119] A "server" refers to a series of computer devices that receive and process information sent by users.

[0120] "Terminal" refers to a device used by a user to input information and check the results (e.g., smartphone, tablet, personal computer).

[0121] "Existing information" refers to past health data and requests that users enter into the system.

[0122] A "database" refers to a collection of information that stores a user's past data and related information.

[0123] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected data, thereby converting it into a format that is easy to analyze.

[0124] "Features" refer to data that extracts specific characteristics or patterns that machine learning models use when generating plans.

[0125] "Generative artificial intelligence" refers to AI technology that generates the optimal plan for the user based on collected data.

[0126] A "plan" refers to specific action guidelines or suggestions created by a generative artificial intelligence system based on user needs.

[0127] "Feedback" refers to the opinions and requests for revisions that users provide regarding the generated plan.

[0128] "Update" refers to the process of generating new plans based on feedback or adjusting existing plans.

[0129] This invention relates to a system that automatically generates and proposes an optimal plan using generative artificial intelligence based on the user's existing information. The user, terminal, and server play important roles as the main components of this system. Specific embodiments are described below.

[0130] System Configuration

[0131] In this system, users input information using a device (e.g., smartphone, tablet, PC), and the device transmits that information to the server. The server receives the information, retrieves relevant information from the database, analyzes the data, and generates the optimal plan for the user.

[0132] Hardware and software to be used

[0133] User terminal: A device used by a user to input information. Specifically, this includes smartphones, tablets, and personal computers.

[0134] Server: A computer device that receives, preprocesses, analyzes, and sends generated plans based on data. Specific software that could be used includes Python, SQL, scikit-learn, pandas, and generative artificial intelligence (e.g., GPT-3®).

[0135] Database: Data storage for storing existing user information and related information. SQL databases are often used.

[0136] Data processing and data calculation

[0137] 1. Receiving user input information

[0138] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input a specific request such as, "I want an exercise and meal plan to lose 5kg."

[0139] 2. Transfer and Reception of Information

[0140] The device sends information collected from the user to the server. This transmission is done via an HTTP request.

[0141] The server receives this request and retrieves relevant information (e.g., past health data or purchase history) from the database. Queries to the database are executed using the SQL language.

[0142] 3. Data preprocessing

[0143] The server preprocesses the collected data. Specific processing includes data cleaning (handling missing or outlier values) and data normalization (scaling). These processes are performed using libraries such as Python's pandas library.

[0144] 4. Feature extraction

[0145] The server inputs preprocessed data into a machine learning model and extracts features. The scikit-learn library is used for the specific processing. For example, the user's weight fluctuation patterns and preferred exercise types might be extracted as features.

[0146] 5. Plan generation using generative artificial intelligence

[0147] The server inputs the extracted features as prompts into a generative artificial intelligence (AI) to generate the optimal plan. The generative AI (e.g., GPT-3) then performs analysis based on these prompts and generates a specific plan.

[0148] 6. Converting and sending the plan format.

[0149] The server converts the generated plan into a specified format (e.g., JSON format) and sends it to the terminal. This process is performed as an HTTP response.

[0150] The device receives this response and notifies the user. Specifically, it notifies the user via smartphone push notifications, etc., that "A new plan has been generated," and displays the plan details within the app.

[0151] Specific example

[0152] For example, by inputting prompt statements like the following into the AI ​​model, it is possible to generate a specific plan based on the user's request:

[0153] Example of a prompt:

[0154] "I want an exercise and meal plan to lose 5kg."

[0155] Based on this prompt, the AI ​​model generates text suggesting a "three-times-a-week jogging and low-fat meal plan," and the server sends that text to the terminal for the user to receive.

[0156] The above describes an embodiment of the present invention, which allows users to quickly obtain individually customized plans and enables dynamic adjustments based on feedback.

[0157] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0158] Step 1:

[0159] User input phase

[0160] Input: Existing user information or specific requests. Example: "I want an exercise and meal plan to lose 5kg."

[0161] Output: Information entered by the user into the terminal

[0162] Specific operation: The user uses an application on their device (smartphone, computer, etc.) to enter their existing information (e.g., current weight, activity level, etc.) and specific requests into a text field, and then presses the "Send" button. This information is temporarily stored on the device.

[0163] Step 2:

[0164] Data collection phase

[0165] Input: User input information saved on the device

[0166] Output: Information to send to the server

[0167] Specific operation: The terminal packages the user's input information as an HTTP request and sends it to the server. The server receives this request and retrieves relevant information such as past health data and purchase history from the user database using SQL queries.

[0168] Step 3:

[0169] Data preprocessing and feature extraction phase

[0170] Input: User information collected by the server, related database information

[0171] Output: Preprocessed data and features

[0172] Specific operation: The server preprocesses the collected data. First, it cleans the data (e.g., removes outliers, imputes missing values), and then normalizes (scales) it. Next, it extracts features using a machine learning model (e.g., scikit-learn). These features include the user's weight change patterns and preferred exercise types.

[0173] Step 4:

[0174] Plan generation phase

[0175] Input: Preprocessed data and extracted features

[0176] Output: Generated plan

[0177] Specific operation: The server inputs the extracted features as prompts into a generative artificial intelligence model (e.g., GPT-3). The generative AI analyzes these prompts and generates a specific plan. For example, it might generate a plan for "jogging three times a week and a low-fat diet."

[0178] Step 5:

[0179] Plan presentation phase

[0180] Input: Generated plan

[0181] Output: Information presented to the user

[0182] Specific operation: The server converts the generated plan into a predetermined format (such as JSON format) and sends it to the device as an HTTP response. The device receives this response and notifies the user. Specifically, it uses smartphone push notifications to inform the user that "a new plan has been generated" and displays the detailed plan within the app.

[0183] Step 6:

[0184] Dialogue and Feedback Phase

[0185] Input: User feedback

[0186] Output: Updated plan

[0187] Specific operation: The user sends questions and feedback about the generated plan through their device. Specifically, they use an input form within the app to enter feedback such as, "Jogging is difficult. Are there any other exercises?" The device sends this feedback to the server as an HTTP request. The server receives the feedback and, if necessary, inputs new prompts into the generative AI to regenerate the plan. The server then sends the regenerated plan to the device and presents it to the user again. For example, it might suggest, "Cycling three times a week instead."

[0188] Through the above processing steps, users can quickly obtain individually customized plans, and the system allows for dynamic adjustments to those plans based on their feedback.

[0189] (Application Example 1)

[0190] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0191] Modern factories are required to maximize production efficiency while minimizing machine downtime. However, conventional methods lack the means to comprehensively analyze the operating data, inspection information, and current work status of each machine to generate optimal production schedules and work plans. As a result, problems such as inefficient production schedules and unplanned downtime occur. This invention aims to solve these problems and improve the production efficiency of factories.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0193] In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and to obtain relevant information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for acquiring machine operation data, inspection information, and current work status in the factory and generating an optimal production schedule and work plan, and means for transmitting the generated production plan as instructions to the factory robots. This makes it possible to improve the overall operational efficiency of the factory and generate an optimal production schedule and work plan that minimizes downtime.

[0194] A "user" is a factory operator who operates the system and inputs the necessary information.

[0195] "Existing information" refers to information that users input into the system, such as product delivery dates, production quantities, and work instructions.

[0196] A "server" is a computer system that receives user information, retrieves related information from a database, and generates the optimal plan.

[0197] A "database" is a system that stores and manages various types of factory data, such as machine operation data, maintenance information, and work history.

[0198] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected information.

[0199] "Features" are important pieces of information extracted from pre-processed data to optimize production schedules.

[0200] "Generative artificial intelligence" refers to an artificial intelligence system that automatically generates the optimal plan based on the input data.

[0201] An "optimal plan" is a production schedule and work plan designed to maximize factory production efficiency and minimize downtime.

[0202] "Prescribed format" refers to the state in which the generated plan has been converted into a format that can be executed by a terminal or robot.

[0203] A "terminal" is a device used by users to input information or to review generated plans.

[0204] "Feedback" refers to requests for revisions or opinions that users provide regarding the generated plan.

[0205] "Machine operation data in a factory" refers to data on the operating status, usage time, and efficiency of machinery within a factory.

[0206] "Inspection information" refers to data on the maintenance history and maintenance plans of machinery within the factory.

[0207] "Current work status" refers to data on ongoing work and production activities within the factory.

[0208] A "production schedule" is a plan of time and work to efficiently manage and operate a factory's production process.

[0209] A "work plan" is a plan that outlines specific work instructions and procedures.

[0210] A "robot" is an automated machine that performs production activities and tasks within a factory.

[0211] This invention is a system that automatically generates and proposes an optimal production schedule and work plan to the user using generative artificial intelligence (generative AI model) based on factory machine operation data, inspection information, and current work status. The embodiments for carrying out this invention will be described in detail below.

[0212] System Configuration

[0213] This system consists of the following main components:

[0214] 1. Terminal as a user input device

[0215] 2. Servers that collect and process data

[0216] 3. Factory robots that perform specified tasks

[0217] Hardware and software used

[0218] Devices as terminals: Smartphones, tablets, personal computers, etc.

[0219] A powerful computer that acts as a server for data collection, preprocessing, feature extraction, and plan generation.

[0220] Machine learning software: The scikit-learn library in Python is used for data preprocessing, and RandomForestRegressor is used as the generative AI model.

[0221] Robot: An automated machine in a factory that performs a specified production activity.

[0222] Processing flow

[0223] 1. User input phase

[0224] The user inputs requirements such as delivery date and production quantity through the terminal interface. For example, a request might be, "I want to produce 1,000 units of the product by December 31st."

[0225] 2. Data Collection Phase

[0226] The server receives machine operation data, inspection information, and current work status collected within the factory.

[0227] 3. Data preprocessing and feature extraction phase

[0228] The server cleans, normalizes, and removes outliers from the collected data. Specifically, it uses the Python scikit-learn library.

[0229] After the data has been preprocessed, features are extracted. For example, these might include machine downtime or operating efficiency.

[0230] 4. Plan Generation Phase

[0231] The server inputs the extracted features into a generating AI model to automatically generate the optimal production schedule and work plan. Here, RandomForestRegressor is used to generate the plan.

[0232] The generated plan might be something like, "a production schedule that maximizes uptime and includes maintenance three times a week."

[0233] 5. Plan Presentation Phase

[0234] The generated plan is converted to a predetermined format on the server and sent to the terminal. The user reviews the plan through the terminal and sends instructions to the factory robots as needed.

[0235] 6. Dialogue and Feedback Phase

[0236] Users provide feedback on the generated plan. For example, they might say, "I'd like to extend the deadline to January 31st."

[0237] The server regenerates the plan based on this feedback and provides the updated plan to the user.

[0238] Specific example

[0239] 1. If the user is a factory operator

[0240] The operator enters a request stating, "We want to produce 1,000 units of product by December 31st."

[0241] The server collects factory machine operation data, inspection information, and current work status, extracts features, and then generates an optimal production schedule.

[0242] For example, a production schedule that maximizes operating time and includes maintenance three times a week is proposed.

[0243] If an operator provides feedback requesting an extension of the delivery date, the server analyzes the feedback and generates a new schedule.

[0244] Example of a prompt

[0245] For example, an operator might input the following information as an example of health data collection:

[0246] We collect the following information as user health data:

[0247] Weight changes over the past 6 months

[0248] Food diary

[0249] How many times a week do you exercise?

[0250] Please generate the optimal plan to lose 5kg.

[0251] Thus, the present invention can maximize factory production efficiency and eliminate inefficient production schedules and unplanned downtime.

[0252] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0253] Step 1:

[0254] The user inputs existing information through the terminal's interface. This input information includes production requests, such as "We want to produce 1000 units of product by December 31st." The input information is sent directly from the terminal to the server.

[0255] Step 2:

[0256] The server receives existing information sent from the terminal. It also retrieves relevant information from the database. This relevant information includes machine operation data, inspection information, and current work status in the factory. The server collects all of this data together.

[0257] Step 3:

[0258] The server preprocesses the collected information. Preprocessing involves cleaning, normalizing, and removing outliers from the data. Specifically, it uses the Python scikit-learn library to scale and filter the data to create an accurate dataset.

[0259] Step 4:

[0260] The server extracts features from the pre-processed data. These features include, for example, the average machine downtime and operating efficiency. These features are important indicators for optimizing production schedules.

[0261] Step 5:

[0262] The server inputs the extracted features into a generating AI model (RandomForestRegressor) to generate the optimal production schedule and work plan. For example, a production schedule that maximizes uptime and includes three maintenance cycles per week might be generated. The generated production plan is temporarily stored on the server.

[0263] Step 6:

[0264] The generated plan is converted to a predetermined format by the server. The converted plan is sent to the terminal and displayed on the user's (factory operator's) screen. The user can review the production plan and modify the work instructions as needed.

[0265] Step 7:

[0266] Users provide feedback through their devices. This feedback may include requests such as, "I would like to extend the delivery date to January 31st." The feedback is sent from the device to the server.

[0267] Step 8:

[0268] The server updates the plan based on the feedback received. The generating AI model analyzes the new feedback information and regenerates the production schedule as needed. For example, a "new production schedule that accommodates the extended delivery date" is generated.

[0269] Step 9:

[0270] The updated plan is resent from the server to the terminal and displayed again to the user. The user reviews the new plan and sends it to the robot as the final work instruction. Once this is complete, the system begins processing the next cycle.

[0271] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0272] This invention relates to a system that combines existing information provided by the user with emotional information read by an emotion engine, automatically generates an optimal plan using generative artificial intelligence, and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[0273] In this system, the user, terminal, server, and emotion engine work together to generate a plan that is tailored to the user's needs and emotional state. The following processes are performed in an embodiment of the invention.

[0274] Program processing

[0275] 1. User Input Phase

[0276] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[0277] 2. Data Collection Phase

[0278] The terminal receives information entered by the user and sends it to the server. This process takes place in real time.

[0279] 3. Emotional Data Collection Phase

[0280] The terminal analyzes the user's emotional state using an emotion engine and transmits the result to the server. The emotion engine determines the emotion the user is currently feeling using face recognition technology and voice analysis technology.

[0281] 4. Data Collection and Preprocessing Phase

[0282] The server receives the user's input information and emotional data and retrieves relevant information from the database. For example, it collects past weight fluctuations, diet records, exercise history, etc.

[0283] The server preprocesses the collected information, performing data cleaning, outlier removal, and incomplete data supplementation. At this stage, the data is converted into a consistent format.

[0284] 5. Feature Extraction Phase

[0285] The server extracts features using the preprocessed data. Using a machine learning model, it identifies important attributes and indicators from the data. For example, it extracts the user's exercise pattern, diet tendency, allergy information, etc.

[0286] 6. Plan Generation Phase

[0287] The server inputs the extracted features and emotional data into a generative artificial intelligence and instructs it to generate a plan suitable for the user's needs and current emotional state.

[0288] The generative artificial intelligence analyzes the input data and automatically generates an optimal plan for the user.

[0289] 7. Plan Presentation Phase

[0290] Based on the analysis results, the generative artificial intelligence generates a specific exercise plan and diet plan. For example, it proposes "Jogging three times a week and a low-fat diet plan".

[0291] The server converts the generated plan and formats it into the required format before sending it to the user's terminal.

[0292] 8. Dialogue and feedback phase based on user emotions

[0293] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[0294] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[0295] The device sends this feedback to the server in real time.

[0296] 9. Feedback Analysis and Plan Re-adjustment Phase

[0297] The server receives user feedback and emotional data determined by the emotion engine, and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[0298] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[0299] The server converts the updated plan and sends it back to the terminal.

[0300] 10. Displaying updated plans

[0301] The device will display the plan to the user again and provide updated information.

[0302] Specific example

[0303] The following process is a concrete example of what happens when a user requests a health plan.

[0304] 1. User Input Phase

[0305] The user uses the terminal to input "I want an exercise and diet plan to lose 5 kg of weight".

[0306] 2. Data Collection Phase

[0307] The terminal sends this input to the server.

[0308] 3. Emotional Data Collection Phase

[0309] The terminal uses the emotion engine to analyze the user's facial expressions and voice, and sends the emotional data to the server.

[0310] 4. Data Collection and Preprocessing Phase

[0311] The server retrieves the user's existing health data (e.g., past weight changes, diet records, exercise history) from the database.

[0312] 5. Feature Extraction Phase

[0313] The server removes outliers from the collected data, normalizes the data, and then extracts features.

[0314] 6. Plan Generation Phase

[0315] The server inputs the features and emotional data into the generative artificial intelligence to generate a plan that is optimal for the user's needs and emotions.

[0316] 7. Plan Presentation Phase

[0317] The generative artificial intelligence generates a plan that proposes "Jogging three times a week and a low-fat diet plan" and sends it to the terminal via the server.

[0318] 8. Dialogue and feedback phase based on user emotions

[0319] The device displays a plan to the user. The user provides feedback on the plan, such as, "Jogging is difficult. Are there any other exercises?"

[0320] The device sends this feedback to the server.

[0321] 9. Feedback Analysis and Plan Re-adjustment Phase

[0322] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate a plan under new conditions.

[0323] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[0324] The server resends the update plan to the device.

[0325] 10. Displaying updated plans

[0326] The device displays the updated plan to the user.

[0327] This invention allows users to receive more personalized plans according to their emotional state, achieving high user satisfaction by flexibly responding to the user's emotions.

[0328] The following describes the processing flow.

[0329] Step 1:

[0330] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[0331] Step 2:

[0332] The terminal receives user input information and sends it to the server.

[0333] Step 3:

[0334] The device uses a built-in emotion engine to determine the user's current emotional state. This emotion determination is performed in real time based on data such as the user's facial expressions and voice.

[0335] Step 4:

[0336] The device sends emotion data, determined by the emotion engine, to the server. This ensures that the user's input information and emotional state are received by the server.

[0337] Step 5:

[0338] The server analyzes the user's input and retrieves relevant information from the database. For example, it collects data such as past weight changes, dietary records, and exercise history.

[0339] Step 6:

[0340] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization.

[0341] Step 7:

[0342] The server uses machine learning models to extract key features from preprocessed data. For example, it identifies the user's past exercise patterns, dietary habits, and allergy information.

[0343] Step 8:

[0344] The server inputs the extracted features and sentiment data into a generative artificial intelligence system, instructing it to generate the optimal plan based on the user's needs and current emotional state.

[0345] Step 9:

[0346] Generative artificial intelligence analyzes input data and automatically generates the optimal plan for the user. For example, it might suggest a plan that includes jogging three times a week and a low-fat diet.

[0347] Step 10:

[0348] The server receives the generated plan, converts it to a predetermined format, and then sends it to the terminal.

[0349] Step 11:

[0350] The device displays the generated plan to the user. For example, it provides a detailed plan through smartphone notifications.

[0351] Step 12:

[0352] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[0353] Step 13:

[0354] The device receives user feedback and sends it to the server in real time.

[0355] Step 14:

[0356] The server analyzes user feedback and sentiment data, and, if necessary, instructs the generative artificial intelligence to generate plans under new conditions.

[0357] Step 15:

[0358] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might suggest "cycling three times a week instead."

[0359] Step 16:

[0360] The server converts the updated plan and sends it back to the terminal.

[0361] Step 17:

[0362] The device displays the updated plan to the user and provides the latest plan information.

[0363] Through these steps, the system responds quickly and accurately to user needs and emotions, generating and adjusting plans dynamically. This allows users to receive a personalized plan that is best suited to their situation and feelings.

[0364] (Example 2)

[0365] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0366] Traditional plan generation systems have struggled to provide personalized plans that take into account the user's emotional state. Therefore, there is a need for a system that generates flexible and optimal plans based on the user's current emotions and updates them in real time based on feedback.

[0367] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving existing information and sentiment information transmitted from the user and obtaining relevant information from a database, means for preprocessing the collected information and performing data cleaning and removal of outliers, and means for extracting features using the preprocessed data. This makes it possible to generate an optimal plan tailored to the user's sentiment state and update it in real time as needed.

[0368] A "user" refers to an individual who uses the system to input information and provides feedback on the proposed plan.

[0369] A "terminal" refers to an electronic device used by a user to input information, send the results of an analysis of their emotional state to a server, and receive a generated plan.

[0370] A "server" refers to a computer system that receives data sent by users, retrieves relevant information from a database, and performs data preprocessing, feature extraction, plan generation, and updates.

[0371] An "emotion engine" refers to an automated tool that analyzes a user's facial expressions and voice to obtain emotional information.

[0372] "Existing information" refers to past data and information about the current situation that users input into the system.

[0373] "Related information" refers to information that the server retrieves from the database, including user health data, past exercise history, and dietary records.

[0374] "Preprocessing" refers to the process of performing operations such as cleaning collected data, removing outliers, and supplementing incomplete data.

[0375] "Features" refer to important attributes or metrics extracted from pre-processed data.

[0376] A "generative artificial intelligence model" refers to a machine learning model that automatically generates the optimal plan based on user needs and emotional information.

[0377] "Plan" refers to exercise and dietary suggestions generated by generative artificial intelligence.

[0378] "Feedback" refers to the opinions and requests that users provide regarding the generated plan.

[0379] "Real-time" refers to a state where input and output occur almost simultaneously, with very little latency.

[0380] This invention relates to a system that combines existing information provided by the user with emotional information acquired by an emotion engine, automatically generates an optimal plan using a generative artificial intelligence model, and proposes it to the user. Below, we will describe the specific program processing of the system in natural language, and add specific examples.

[0381] System Configuration

[0382] This system consists of several main components, including the user, terminal, server, emotion engine, and generative artificial intelligence model. The roles and interactions of each component enable the system to automatically generate and present plans based on the user's needs and emotional state.

[0383] Hardware and software configuration

[0384] 1. Terminal:

[0385] These are devices that provide a user interface, such as smartphones, tablets, or PCs.

[0386] The built-in camera and microphone are used to analyze facial expressions and voice using an emotion engine.

[0387] The interface operates within a web browser or a specific application.

[0388] 2. Server:

[0389] It is a high-performance computer system that collects and preprocesses user data and generates plans using a generative artificial intelligence model.

[0390] It has a built-in database that stores and manages relevant information such as the user's health data, past exercise history, and dietary records.

[0391] 3. Emotional Engine:

[0392] This software uses computer vision and speech analysis technologies to analyze the user's emotional state.

[0393] For example, emotions can be identified from the user's facial expressions and voice tone.

[0394] 4. Generative artificial intelligence models:

[0395] For example, advanced generative artificial intelligence such as GPT-3 is used to analyze collected data and generate the optimal plan for the user.

[0396] Specific examples of actions

[0397] 1. User input phase:

[0398] The user enters their desired goals and requirements through the terminal's interface. For example, they might enter, "I want an exercise and meal plan to lose 5kg."

[0399] 2. Data Collection Phase:

[0400] The terminal receives information entered by the user and transmits this information to the server in real time.

[0401] 3. Emotional Data Collection Phase:

[0402] The device uses its built-in emotion engine to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[0403] 4. Data processing and preprocessing:

[0404] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database and performs preprocessing. The collected data is cleaned, outliers are removed, and the format is standardized.

[0405] 5. Feature extraction:

[0406] The server extracts features from the pre-processed data and identifies important attributes and metrics. For example, it uses machine learning algorithms to identify exercise patterns, dietary tendencies, allergy information, and so on.

[0407] 6. Plan generation:

[0408] The server inputs extracted features and sentiment data into a generative artificial intelligence model, instructing it to generate an optimal plan based on the user's needs and emotional state. The generative AI model analyzes this data and generates, for example, a "three-times-a-week-jogging and low-fat-meal plan."

[0409] 7. Presentation of the plan and feedback:

[0410] The generative artificial intelligence model sends the generated plan to the terminal via the server, and the terminal displays this plan to the user. The user then provides feedback on this plan.

[0411] The device sends user feedback to the server in real time, and the server analyzes the feedback and sentiment data.

[0412] 8. Readjust the plan:

[0413] The server sends the analysis results to a generative artificial intelligence model and instructs it to generate a plan under new conditions. For example, it might generate a "cycling plan three times a week instead."

[0414] Examples of prompt statements

[0415] For example, the following request is input to a generative artificial intelligence model:

[0416] "I'd like you to suggest an exercise and meal plan to lose 5kg. My current weight is 70kg, and my past exercise history mainly consists of jogging, but I don't have any specific dietary restrictions."

[0417] Through the above process, the present invention makes it possible to provide personalized plans according to the user's emotional state, thereby improving user satisfaction.

[0418] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0419] Step 1:

[0420] User input phase

[0421] The user enters their desired goals and requirements through the terminal's interface.

[0422] Specific input: The user enters, "I would like an exercise and meal plan to lose 5kg."

[0423] The device receives this information and stores it in its internal data structure.

[0424] Input: User's goals and requirements

[0425] Output: Saved user input data

[0426] Step 2:

[0427] Data collection phase

[0428] The terminal sends the information received from the user to the server.

[0429] Specific operation: The terminal sends saved user input data to the server in real time.

[0430] Input: User input data

[0431] Output: User data sent to the server

[0432] Step 3:

[0433] Emotional data collection phase

[0434] The device uses an emotion engine to analyze the user's facial expressions and voice to acquire emotion data.

[0435] Specific operation: Activates the facial recognition camera and microphone, processes the collected data with an emotion engine, and generates emotion data.

[0436] The device sends the acquired emotion data to the server.

[0437] Input: User's facial expression data and voice data

[0438] Output: Sentiment data sent to the server

[0439] Step 4:

[0440] Data acquisition and preprocessing phase

[0441] The server receives existing information and sentiment data sent by the user and retrieves relevant health data from the database.

[0442] Specific actions: Query the database to collect data such as past weight fluctuations, meal records, and exercise history.

[0443] The server preprocesses the collected data, cleaning it, removing outliers, and completing incomplete data.

[0444] Input: Existing user information and sentiment data

[0445] Output: Preprocessed data

[0446] Step 5:

[0447] Feature extraction phase

[0448] The server extracts features using the pre-processed data.

[0449] Specific operation: Machine learning algorithms (e.g., random forest, decision tree, etc.) are used to identify important attributes such as exercise patterns, dietary tendencies, and allergy information.

[0450] Input: Pre-processed data

[0451] Output: Extracted features

[0452] Step 6:

[0453] Plan generation phase

[0454] The server inputs the extracted feature vectors and sentiment data into a generative artificial intelligence model and instructs it to generate the optimal plan.

[0455] Specific operation: A generative artificial intelligence model (e.g., GPT-3) is input with prompt text and feature data to generate a plan.

[0456] Input: Extracted features and sentiment data

[0457] Output: Generated plan

[0458] Step 7:

[0459] Plan presentation phase

[0460] The generative artificial intelligence model sends the generated plan back to the server, which then formats the plan into a predetermined format.

[0461] Specific actions: Convert the generated plan to text format and attach additional information as needed.

[0462] The server sends the plan to the terminal.

[0463] Input: Generated plan

[0464] Output: Plan sent to the terminal

[0465] Step 8:

[0466] Dialogue and feedback phase based on user emotions

[0467] The device displays the plan generated for the user.

[0468] Specific actions: The plan will be displayed on the smartphone or PC screen, and the user will be notified.

[0469] Users provide feedback on the plan and enter it into their device. They might ask questions like, "Jogging is difficult. Are there any other exercises?"

[0470] The device sends this feedback to the server in real time.

[0471] Input: User feedback

[0472] Output: Feedback sent to the server

[0473] Step 9:

[0474] Feedback analysis and plan readjustment phase

[0475] The server analyzes user feedback and sentiment data.

[0476] Specific operation: Analyze feedback data and instruct the generative AI model to generate a plan under new conditions. Example: Generate a new plan that suggests cycling three times a week instead.

[0477] The generative artificial intelligence model generates a plan based on new conditions and sends it to the server.

[0478] Input: Feedback and sentiment data

[0479] Output: Updated plan

[0480] Step 10:

[0481] Display updated plans

[0482] The server sends the updated plan to the device.

[0483] The device displays the updated plan to the user.

[0484] Specific action: The updated plan will be displayed again on the smartphone or PC screen.

[0485] Input: Updated plan

[0486] Output: Plan displayed to the user

[0487] (Application Example 2)

[0488] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0489] Previously, the lack of a system to quickly provide personalized plans tailored to users' emotions and needs resulted in low user satisfaction. Furthermore, real-time product recommendations in physical stores were difficult, leading to an unoptimized shopping experience. To address these challenges, a system that understands users' emotional states and provides immediate, appropriate recommendations based on that understanding is essential.

[0490] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for analyzing the user's emotional state using an emotion engine and transmitting the emotional information to the server, and means for displaying optimal product suggestions to the user in the store through a smartphone or smart glasses interface. This enables real-time and personalized product suggestions that correspond to the user's emotional state.

[0491] "Existing information" refers to data and historical information that the user has provided in advance.

[0492] A "terminal" refers to a device used by a user to input or receive information through an interface. Examples include smartphones and smart glasses.

[0493] A "server" refers to a computer system that processes data sent by users and retrieves and manages related information.

[0494] A "database" refers to a system in which information is stored and managed.

[0495] "Preprocessing" refers to the process of cleansing and normalizing collected data and converting it into a format from which features can be extracted.

[0496] "Features" refer to important attributes and indicators extracted from data, which form the basis for analysis by machine learning models.

[0497] "Generative artificial intelligence" refers to machine learning and artificial intelligence technologies that automatically generate the optimal plan based on collected data and features.

[0498] A "plan" refers to a specific action plan or suggestion proposed by a generative artificial intelligence system based on the user's needs and emotions.

[0499] An "emotion engine" refers to a technology that analyzes a user's facial expressions and voice to determine their emotional state.

[0500] An "interface" refers to the screen or input device that a user uses to interact with a system.

[0501] "Feedback" refers to the reactions and opinions that users provide regarding the system.

[0502] A "smartphone" refers to a mobile communication device that has calling capabilities and a wide range of applications.

[0503] "Smart glasses" refer to a type of wearable device, specifically glasses with a display function, that presents information to the user.

[0504] "Product recommendations" refer to specific product information that is recommended based on the user's needs and emotional state.

[0505] A "physical store" refers to a store that has a physical presence and is a place where users can go in person to purchase products.

[0506] This invention relates to a system that provides personalized in-store product recommendations based on the user's emotional state. The specific configuration and operation of the system are described below.

[0507] Hardware and software to be used

[0508] hardware

[0509] 1. Device: A device used by the user (e.g., smartphone, smart glasses). This includes cameras and microphones.

[0510] 2. Server: A computer system that processes, manages, and runs generative artificial intelligence.

[0511] 3. Database: Stores user information, past purchase history, sentiment data, etc.

[0512] software

[0513] 1. Emotion Engine: A technology that analyzes a user's facial expressions and voice to determine their emotional state (e.g., FaceAPI, Microsoft® Emotion API).

[0514] 2. Generative Artificial Intelligence: Machine learning and artificial intelligence technologies that automatically generate the optimal plan based on collected data (e.g., GPT-4®).

[0515] 3. Database management software: A system for operating and managing databases (e.g., MySQL®).

[0516] 4. Machine learning frameworks: Used for data preprocessing, feature extraction, model building, and analysis (e.g., Scikit-learn, TENSORFLOW®).

[0517] System operation

[0518] 1. User input

[0519] Users input their current shopping purpose and product categories of interest via their smartphone or smart glasses.

[0520] Specific example: The user enters "I want new summer clothes" into the device.

[0521] 2. Submitting the entered information

[0522] The terminal sends user input information to the server.

[0523] 3. Analysis of emotional state

[0524] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[0525] Specific example: Determining a user's level of excitement and interest based on their facial expressions and comments while viewing products in a store.

[0526] 4. Data Collection and Preprocessing

[0527] The server receives user input information and sentiment data, retrieves relevant data (such as past purchase history) from the database, and preprocesses that data. Preprocessing includes data cleansing, normalization, and consistency checks.

[0528] 5. Feature Extraction

[0529] The server extracts key features from pre-processed data using a machine learning model. These features represent the user's purchasing tendencies and current emotional state.

[0530] 6. Generating the optimal plan

[0531] The server inputs extracted feature data and sentiment data into a generative artificial intelligence system to generate an optimal product suggestion plan for the user. The generated plan is then converted by the server into a predetermined format and sent to the terminal.

[0532] Specific example: A plan is proposed that says, "New summer casual dresses are now 20% off."

[0533] 7. Plan provision and feedback

[0534] The terminal displays a proposed plan to the user, and the user provides feedback on it. The feedback is sent to the server, which updates the plan based on that feedback.

[0535] Specific example: A user enters feedback saying, "Is this dress available in a different color?"

[0536] Example of a prompt

[0537] "I want some new summer clothes."

[0538] "I'd like to know about any discounts on the products I've shown interest in."

[0539] "I'd like to check if this product is available in other colors."

[0540] These methods enable real-time and personalized product recommendations tailored to the user's emotional state, thereby increasing user satisfaction.

[0541] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0542] Step 1:

[0543] The user uses a device (smartphone or smart glasses) to input their current shopping purpose and product categories of interest. The user's request, "I want new summer clothes," is generated as input data.

[0544] Step 2:

[0545] The terminal sends user input information to the server. The terminal sends a request, which is the input data, and the server receives it. Data transmission occurs in real time.

[0546] Step 3:

[0547] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and generates emotion data. This emotion data includes emotional information (for example, excitement) determined from the user's facial expressions and statements when viewing a product.

[0548] Step 4:

[0549] The device sends the generated emotion data to the server. The emotion data is sent from the device to the server and used as input information for analysis.

[0550] Step 5:

[0551] The server receives user input information and sentiment data, and retrieves relevant information from the database. This relevant information includes data indicating the user's past purchase history, product preferences, and current sentiment state.

[0552] Step 6:

[0553] The server performs preprocessing on the collected information, removing outliers and normalizing the data. The preprocessed data is then formatted to ensure data consistency and allow for feature extraction.

[0554] Step 7:

[0555] The server inputs pre-processed data into a machine learning model and extracts features. These features include user purchasing tendencies and product categories of interest. Input data (pre-processed data) and output data (features) are generated at this stage.

[0556] Step 8:

[0557] The server inputs feature data and sentiment data into a generative artificial intelligence system to generate an optimal product recommendation plan. The generated plan is based on the user's needs and emotional state. For example, a plan such as "New summer casual dresses, now 20% off" might be generated.

[0558] Step 9:

[0559] The server converts the generated product proposal plan into a predetermined format and sends it to the terminal. The input data (generated plan) is converted into output data (plan in a predetermined format) and sent.

[0560] Step 10:

[0561] The device displays a suggested plan to the user. The user provides feedback on it (e.g., "Is this dress available in a different color?"). This feedback is then generated as new input data.

[0562] Step 11:

[0563] The device sends user feedback to the server in real time. Feedback data is sent and received by the server. This feedback data is then input as new conditions.

[0564] Step 12:

[0565] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate plans under new conditions. The generative AI generates new product suggestion plans that match the feedback and emotional state. For example, it might generate a product that suggests cycling three times a week instead.

[0566] Step 13:

[0567] The server converts the updated plan into the specified format and resends it to the terminal. The updated plan is then presented to the user.

[0568] Step 14:

[0569] The device displays updated product plans to the user. This allows the user to see new suggestions and have a more satisfying shopping experience.

[0570] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0571] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0572] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0573] [Second Embodiment]

[0574] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0575] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0576] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0577] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0578] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0579] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0580] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0581] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0582] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0583] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0584] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0585] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0586] This invention relates to a system that automatically generates an optimal plan using generative artificial intelligence based on the user's existing information and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[0587] In this system, the user, terminal, and server work together to generate a plan tailored to the user's needs. The following processes are performed in order to implement the invention.

[0588] Program processing

[0589] 1. User Input Phase

[0590] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input, "I want an exercise and meal plan to lose 5kg."

[0591] 2. Data Collection Phase

[0592] The terminal sends the user's input information to the server.

[0593] The server receives information sent by the user and retrieves related information from the database (e.g., past health data, purchase history, etc.).

[0594] 3. Data preprocessing and feature extraction phase

[0595] The server preprocesses the collected information. Preprocessing includes data cleaning, normalization, and removal of outliers.

[0596] The server extracts features from the pre-processed data by running it through a machine learning model. For example, these features might include "the user's weight fluctuation patterns and exercise preferences."

[0597] 4. Plan Generation Phase

[0598] The server inputs the extracted features into a generative artificial intelligence system and instructs it to generate the optimal plan for the user.

[0599] Generative artificial intelligence analyzes the input data and generates the optimal plan for the user. For example, it might recommend a plan that includes jogging three times a week and a low-fat diet.

[0600] 5. Plan Presentation Phase

[0601] The server converts the generated plan into a predetermined format and sends it to the terminal.

[0602] The device will display the plan to the user. For example, the user could view the detailed plan via a smartphone notification.

[0603] 6. Dialogue and Feedback Phase

[0604] Users ask questions and provide feedback about the generated plan. For example, they might say, "Jogging is difficult. Are there any other exercises?"

[0605] The device sends this feedback to the server.

[0606] The server analyzes the feedback and regenerates the plan as needed. Generative artificial intelligence generates a new plan based on the feedback, and the server sends it to the terminal. For example, it might suggest "cycling three times a week instead."

[0607] Specific example

[0608] Let's explain using a specific example of a user requesting a health plan.

[0609] 1. User input phase:

[0610] The user uses their device to input, "I want an exercise and meal plan to lose 5kg."

[0611] 2. Data Collection Phase:

[0612] The terminal sends this input to the server.

[0613] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database.

[0614] 3. Data preprocessing and feature extraction phase:

[0615] The server removes outliers from the collected data and normalizes the data.

[0616] The server uses machine learning models to extract features from the user's health data, such as weight fluctuation patterns and preferred exercise types.

[0617] 4. Plan generation phase:

[0618] The server inputs these features into a generative artificial intelligence system and instructs it to generate a plan that suits the user's needs.

[0619] The generative artificial intelligence generates a plan recommending "jogging three times a week and a low-fat diet."

[0620] 5. Plan presentation phase:

[0621] The server sends the generated plan to the terminal.

[0622] The device notifies the user and displays details within the app.

[0623] 6. Dialogue and Feedback Phase:

[0624] Users send feedback through their devices, saying things like, "Jogging is difficult. Are there any other exercises?"

[0625] The device sends this feedback to the server.

[0626] The server receives feedback and instructs the generative artificial intelligence to generate new movement options.

[0627] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[0628] The server sends the updated plan to the terminal and presents it to the user again.

[0629] As described above, the present invention allows users to receive plans based on their needs quickly and accurately. Furthermore, the plan can be dynamically adjusted through feedback, achieving high flexibility and user satisfaction.

[0630] The following describes the processing flow.

[0631] Step 1:

[0632] The user enters their existing information and specific requests through the terminal's interface. For example, they might enter a request such as "an exercise and meal plan to lose 5kg."

[0633] Step 2:

[0634] The terminal receives information entered by the user and sends that information to the server. This process takes place in real time.

[0635] Step 3:

[0636] After receiving user input, the server retrieves relevant detailed information from the database. For example, it collects data such as past weight fluctuations, dietary records, and exercise history.

[0637] Step 4:

[0638] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization. At this stage, the data is transformed into a consistent format.

[0639] Step 5:

[0640] The server extracts features using pre-processed data. Machine learning models are used to identify important attributes and metrics from the data. For example, it extracts user exercise patterns, dietary habits, and allergy information.

[0641] Step 6:

[0642] The server inputs the extracted features into the generative artificial intelligence and instructs it to generate a plan that suits the user's needs. The generative AI analyzes the input data and automatically generates the optimal plan for the user.

[0643] Step 7:

[0644] Generative artificial intelligence generates specific exercise and meal plans based on the analysis results. For example, it might suggest a plan for jogging three times a week and a low-fat diet.

[0645] Step 8:

[0646] The server receives the generated plan and converts it to a predetermined format before sending it to the user's terminal. This makes the plan easier for the user to understand.

[0647] Step 9:

[0648] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[0649] Step 10:

[0650] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[0651] Step 11:

[0652] The device sends user feedback to the server in real time.

[0653] Step 12:

[0654] The server receives user feedback and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[0655] Step 13:

[0656] Generative artificial intelligence incorporates user feedback to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[0657] Step 14:

[0658] The server receives the updated plan and sends it back to the device.

[0659] Step 15:

[0660] The device will display the plan to the user again and provide updated information.

[0661] Through these steps, the system responds quickly and accurately to user needs, generating and adjusting plans dynamically. This allows users to continuously receive individually customized plans.

[0662] (Example 1)

[0663] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0664] In modern society, health management and the provision of personalized plans are crucial issues. However, existing methods struggle to effectively handle large amounts of data and quickly generate personalized plans. Furthermore, they lack the functionality to dynamically adjust plans based on user feedback. This can lead to users not receiving the most suitable plan, resulting in decreased satisfaction.

[0665] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0666] In this invention, the server includes means for the user to input existing information, means for a terminal to receive existing information transmitted from the user and transfer it to the server, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the plan generated by the generative artificial intelligence into a predetermined format and transmit it to the terminal, means for the terminal to notify and display the plan to the user, means for the user to provide feedback and for the terminal to send the feedback to the server, and means for the server to update the plan based on the feedback. As a result, the user can obtain a personalized plan quickly and accurately, and since the plan is dynamically adjusted based on the feedback, user satisfaction can be increased.

[0667] A "user" refers to an individual who uses the system to request health management and plan generation.

[0668] A "server" refers to a series of computer devices that receive and process information sent by users.

[0669] "Terminal" refers to a device used by a user to input information and check the results (e.g., smartphone, tablet, personal computer).

[0670] "Existing information" refers to past health data and requests that users enter into the system.

[0671] A "database" refers to a collection of information that stores a user's past data and related information.

[0672] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected data, thereby converting it into a format that is easy to analyze.

[0673] "Features" refer to data that extracts specific characteristics or patterns that machine learning models use when generating plans.

[0674] "Generative artificial intelligence" refers to AI technology that generates the optimal plan for the user based on collected data.

[0675] A "plan" refers to specific action guidelines or suggestions created by a generative artificial intelligence system based on user needs.

[0676] "Feedback" refers to the opinions and requests for revisions that users provide regarding the generated plan.

[0677] "Update" refers to the process of generating new plans based on feedback or adjusting existing plans.

[0678] This invention relates to a system that automatically generates and proposes an optimal plan using generative artificial intelligence based on the user's existing information. The user, terminal, and server play important roles as the main components of this system. Specific embodiments are described below.

[0679] System Configuration

[0680] In this system, users input information using a device (e.g., smartphone, tablet, PC), and the device transmits that information to the server. The server receives the information, retrieves relevant information from the database, analyzes the data, and generates the optimal plan for the user.

[0681] Hardware and software to be used

[0682] User terminal: A device used by a user to input information. Specifically, this includes smartphones, tablets, and personal computers.

[0683] Server: A computer device that receives, preprocesses, analyzes, and sends generated plans based on data. Specific software options include Python, SQL, scikit-learn, pandas, and generative artificial intelligence (e.g., GPT-3).

[0684] Database: Data storage for storing existing user information and related information. SQL databases are often used.

[0685] Data processing and data calculation

[0686] 1. Receiving user input information

[0687] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input a specific request such as, "I want an exercise and meal plan to lose 5kg."

[0688] 2. Transfer and Reception of Information

[0689] The device sends information collected from the user to the server. This transmission is done via an HTTP request.

[0690] The server receives this request and retrieves relevant information (e.g., past health data or purchase history) from the database. Queries to the database are executed using the SQL language.

[0691] 3. Data preprocessing

[0692] The server preprocesses the collected data. Specific processing includes data cleaning (handling missing or outlier values) and data normalization (scaling). These processes are performed using libraries such as Python's pandas library.

[0693] 4. Feature extraction

[0694] The server inputs preprocessed data into a machine learning model and extracts features. The scikit-learn library is used for the specific processing. For example, features such as the user's weight fluctuation patterns and preferred exercise types might be extracted.

[0695] 5. Plan generation using generative artificial intelligence

[0696] The server inputs the extracted features as prompts into a generative artificial intelligence (AI) to generate the optimal plan. The generative AI (e.g., GPT-3) then performs analysis based on these prompts and generates a specific plan.

[0697] 6. Converting and sending the plan format.

[0698] The server converts the generated plan into a specified format (e.g., JSON format) and sends it to the terminal. This process is performed as an HTTP response.

[0699] The device receives this response and notifies the user. Specifically, it notifies the user via smartphone push notifications, etc., that "A new plan has been generated," and displays the plan details within the app.

[0700] Specific example

[0701] For example, by inputting prompt statements like the following into the AI ​​model, it is possible to generate a specific plan based on the user's request:

[0702] Example of a prompt:

[0703] "I want an exercise and meal plan to lose 5kg."

[0704] Based on this prompt, the AI ​​model generates text suggesting a "three-times-a-week-jogging and low-fat meal plan," and the server sends that text to the terminal for the user to receive.

[0705] The above describes an embodiment of the present invention, which allows users to quickly obtain individually customized plans and enables dynamic adjustments based on feedback.

[0706] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0707] Step 1:

[0708] User input phase

[0709] Input: Existing user information or specific requests. Example: "I want an exercise and meal plan to lose 5kg."

[0710] Output: Information entered by the user into the terminal

[0711] Specific operation: The user uses an application on their device (smartphone, computer, etc.) to enter their existing information (e.g., current weight, activity level, etc.) and specific requests into a text field, and then presses the "Send" button. This information is temporarily stored on the device.

[0712] Step 2:

[0713] Data collection phase

[0714] Input: User input information saved on the device

[0715] Output: Information to send to the server

[0716] Specific operation: The terminal packages the user's input information as an HTTP request and sends it to the server. The server receives this request and retrieves relevant information such as past health data and purchase history from the user database using SQL queries.

[0717] Step 3:

[0718] Data preprocessing and feature extraction phase

[0719] Input: User information collected by the server, related database information

[0720] Output: Preprocessed data and features

[0721] Specific operation: The server preprocesses the collected data. First, it cleans the data (e.g., removes outliers, imputes missing values), and then normalizes (scales) it. Next, it extracts features using a machine learning model (e.g., scikit-learn). These features include the user's weight change patterns and preferred exercise types.

[0722] Step 4:

[0723] Plan generation phase

[0724] Input: Preprocessed data and extracted features

[0725] Output: Generated plan

[0726] Specific operation: The server inputs the extracted features as prompts into a generative artificial intelligence model (e.g., GPT-3). The generative AI analyzes these prompts and generates a specific plan. For example, it might generate a plan for "jogging three times a week and a low-fat diet."

[0727] Step 5:

[0728] Plan presentation phase

[0729] Input: Generated plan

[0730] Output: Information presented to the user

[0731] Specific operation: The server converts the generated plan into a predetermined format (such as JSON format) and sends it to the device as an HTTP response. The device receives this response and notifies the user. Specifically, it uses smartphone push notifications to inform the user that "a new plan has been generated" and displays the detailed plan within the app.

[0732] Step 6:

[0733] Dialogue and Feedback Phase

[0734] Input: User feedback

[0735] Output: Updated plan

[0736] Specific operation: The user sends questions and feedback about the generated plan through their device. Specifically, they use an input form within the app to enter feedback such as, "Jogging is difficult. Are there any other exercises?" The device sends this feedback to the server as an HTTP request. The server receives the feedback and, if necessary, inputs new prompts into the generative AI to regenerate the plan. The server then sends the regenerated plan to the device and presents it to the user again. For example, it might suggest, "Cycling three times a week instead."

[0737] Through the above processing steps, users can quickly obtain individually customized plans, and the system allows for dynamic adjustments to those plans based on their feedback.

[0738] (Application Example 1)

[0739] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0740] Modern factories are required to maximize production efficiency while minimizing machine downtime. However, conventional methods lack the means to comprehensively analyze the operating data, inspection information, and current work status of each machine to generate optimal production schedules and work plans. As a result, problems such as inefficient production schedules and unplanned downtime occur. This invention aims to solve these problems and improve the production efficiency of factories.

[0741] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0742] In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and to obtain relevant information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for acquiring machine operation data, inspection information, and current work status in the factory and generating an optimal production schedule and work plan, and means for transmitting the generated production plan as instructions to the factory robots. This makes it possible to improve the overall operational efficiency of the factory and generate an optimal production schedule and work plan that minimizes downtime.

[0743] A "user" is a factory operator who operates the system and inputs the necessary information.

[0744] "Existing information" refers to information that users input into the system, such as product delivery dates, production quantities, and work instructions.

[0745] A "server" is a computer system that receives user information, retrieves related information from a database, and generates the optimal plan.

[0746] A "database" is a system that stores and manages various types of factory data, such as machine operation data, maintenance information, and work history.

[0747] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected information.

[0748] "Features" are important pieces of information extracted from pre-processed data to optimize production schedules.

[0749] "Generative artificial intelligence" refers to an artificial intelligence system that automatically generates the optimal plan based on the input data.

[0750] An "optimal plan" is a production schedule and work plan designed to maximize factory production efficiency and minimize downtime.

[0751] "Prescribed format" refers to the state in which the generated plan has been converted into a format that can be executed by a terminal or robot.

[0752] A "terminal" is a device used by users to input information or to review generated plans.

[0753] "Feedback" refers to requests for revisions or opinions that users provide regarding the generated plan.

[0754] "Machine operation data in a factory" refers to data on the operating status, usage time, and efficiency of machinery within a factory.

[0755] "Inspection information" refers to data on the maintenance history and maintenance plans of machinery within the factory.

[0756] "Current work status" refers to data on ongoing work and production activities within the factory.

[0757] A "production schedule" is a plan of time and work to efficiently manage and operate a factory's production process.

[0758] A "work plan" is a plan that outlines specific work instructions and procedures.

[0759] A "robot" is an automated machine that performs production activities and tasks within a factory.

[0760] This invention is a system that automatically generates and proposes an optimal production schedule and work plan to the user using generative artificial intelligence (generative AI model) based on factory machine operation data, inspection information, and current work status. The embodiments for carrying out this invention will be described in detail below.

[0761] System Configuration

[0762] This system consists of the following main components:

[0763] 1. Terminal as a user input device

[0764] 2. Servers that collect and process data

[0765] 3. Factory robots that perform specified tasks

[0766] Hardware and software used

[0767] Devices as terminals: Smartphones, tablets, personal computers, etc.

[0768] A powerful computer that acts as a server for data collection, preprocessing, feature extraction, and plan generation.

[0769] Machine learning software: The scikit-learn library in Python is used for data preprocessing, and RandomForestRegressor is used as the generative AI model.

[0770] Robot: An automated machine in a factory that performs a specified production activity.

[0771] Processing flow

[0772] 1. User input phase

[0773] The user inputs requirements such as delivery date and production quantity through the terminal interface. For example, a request might be, "I want to produce 1,000 units of the product by December 31st."

[0774] 2. Data Collection Phase

[0775] The server receives machine operation data, inspection information, and current work status collected within the factory.

[0776] 3. Data preprocessing and feature extraction phase

[0777] The server cleans, normalizes, and removes outliers from the collected data. Specifically, it uses the Python scikit-learn library.

[0778] After the data has been preprocessed, features are extracted. For example, these might include machine downtime or operating efficiency.

[0779] 4. Plan Generation Phase

[0780] The server inputs the extracted features into a generating AI model to automatically generate the optimal production schedule and work plan. Here, RandomForestRegressor is used to generate the plan.

[0781] The generated plan might be something like, "a production schedule that maximizes uptime and includes maintenance three times a week."

[0782] 5. Plan Presentation Phase

[0783] The generated plan is converted to a predetermined format on the server and sent to the terminal. The user reviews the plan through the terminal and sends instructions to the factory robots as needed.

[0784] 6. Dialogue and Feedback Phase

[0785] Users provide feedback on the generated plan. For example, they might say, "I'd like to extend the deadline to January 31st."

[0786] The server regenerates the plan based on this feedback and provides the updated plan to the user.

[0787] Specific example

[0788] 1. If the user is a factory operator

[0789] The operator enters a request stating, "We want to produce 1,000 units of product by December 31st."

[0790] The server collects factory machine operation data, inspection information, and current work status, extracts features, and then generates an optimal production schedule.

[0791] For example, a production schedule that maximizes operating time and includes maintenance three times a week is proposed.

[0792] If an operator provides feedback requesting an extension of the delivery date, the server analyzes the feedback and generates a new schedule.

[0793] Example of a prompt

[0794] For example, an operator might input the following information as an example of health data collection:

[0795] We collect the following information as user health data:

[0796] Weight changes over the past 6 months

[0797] Food diary

[0798] How many times a week do you exercise?

[0799] Please generate the optimal plan to lose 5kg.

[0800] Thus, the present invention can maximize factory production efficiency and eliminate inefficient production schedules and unplanned downtime.

[0801] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0802] Step 1:

[0803] The user inputs existing information through the terminal's interface. This input information includes production requests, such as "We want to produce 1000 units of product by December 31st." The input information is sent directly from the terminal to the server.

[0804] Step 2:

[0805] The server receives existing information sent from the terminal. It also retrieves relevant information from the database. This relevant information includes machine operation data, inspection information, and current work status in the factory. The server collects all of this data together.

[0806] Step 3:

[0807] The server preprocesses the collected information. Preprocessing involves cleaning, normalizing, and removing outliers from the data. Specifically, it uses the Python scikit-learn library to scale and filter the data to create an accurate dataset.

[0808] Step 4:

[0809] The server extracts features from the pre-processed data. These features include, for example, the average machine downtime and operating efficiency. These features are important indicators for optimizing production schedules.

[0810] Step 5:

[0811] The server inputs the extracted features into a generating AI model (RandomForestRegressor) to generate the optimal production schedule and work plan. For example, a production schedule that maximizes uptime and includes three maintenance cycles per week is generated. The generated production plan is temporarily stored on the server.

[0812] Step 6:

[0813] The generated plan is converted to a predetermined format by the server. The converted plan is sent to the terminal and displayed on the user's (factory operator's) screen. The user can review the production plan and modify the work instructions as needed.

[0814] Step 7:

[0815] Users provide feedback through their devices. This feedback may include requests such as, "I would like to extend the delivery date to January 31st." The feedback is sent from the device to the server.

[0816] Step 8:

[0817] The server updates the plan based on the feedback received. The generating AI model analyzes the new feedback information and regenerates the production schedule as needed. For example, a "new production schedule that accommodates the extended delivery date" is generated.

[0818] Step 9:

[0819] The updated plan is resent from the server to the terminal and displayed again to the user. The user reviews the new plan and sends it to the robot as the final work instruction. Once this is complete, the system begins processing the next cycle.

[0820] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0821] This invention relates to a system that combines existing information provided by the user with emotional information read by an emotion engine, automatically generates an optimal plan using generative artificial intelligence, and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[0822] In this system, the user, terminal, server, and emotion engine work together to generate a plan that is tailored to the user's needs and emotional state. The following processes are performed in an embodiment of the invention.

[0823] Program processing

[0824] 1. User Input Phase

[0825] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[0826] 2. Data Collection Phase

[0827] The terminal receives information entered by the user and sends it to the server. This process takes place in real time.

[0828] 3. Emotional Data Collection Phase

[0829] The device analyzes the user's emotional state using an emotion engine and sends the results to the server. The emotion engine uses facial recognition and voice analysis technologies to determine the emotions the user is currently feeling.

[0830] 4. Data Acquisition and Preprocessing Phase

[0831] The server receives user input information and sentiment data, and retrieves relevant information from the database. For example, it collects past weight fluctuations, food records, and exercise history.

[0832] The server preprocesses the collected information, cleaning the data, removing outliers, and completing incomplete data. At this stage, the data is converted into a consistent format.

[0833] 5. Feature Extraction Phase

[0834] The server extracts features using pre-processed data. Machine learning models are used to identify important attributes and metrics from the data. For example, it might extract user exercise patterns, dietary habits, and allergy information.

[0835] 6. Plan Generation Phase

[0836] The server inputs the extracted features and sentiment data into a generative artificial intelligence system and instructs it to generate a plan that is suitable for the user's needs and current emotional state.

[0837] Generative artificial intelligence analyzes the input data and automatically generates the optimal plan for the user.

[0838] 7. Plan Presentation Phase

[0839] Generative artificial intelligence generates specific exercise and meal plans based on the analysis results. For example, it might suggest a plan for jogging three times a week and a low-fat diet.

[0840] The server converts the generated plan and formats it into the required format before sending it to the user's terminal.

[0841] 8. Dialogue and feedback phase based on user emotions

[0842] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[0843] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[0844] The device sends this feedback to the server in real time.

[0845] 9. Feedback Analysis and Plan Re-adjustment Phase

[0846] The server receives user feedback and emotional data determined by the emotion engine, and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[0847] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[0848] The server converts the updated plan and sends it back to the terminal.

[0849] 10. Displaying updated plans

[0850] The device will display the plan to the user again and provide updated information.

[0851] Specific example

[0852] The following process is a concrete example of what happens when a user requests a health plan.

[0853] 1. User Input Phase

[0854] The user uses their device to input, "I want an exercise and meal plan to lose 5kg."

[0855] 2. Data Collection Phase

[0856] The terminal sends this input to the server.

[0857] 3. Emotional Data Collection Phase

[0858] The device uses an emotion engine to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[0859] 4. Data Acquisition and Preprocessing Phase

[0860] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database.

[0861] 5. Feature Extraction Phase

[0862] The server removes outliers from the collected data and normalizes the data. Next, it extracts features.

[0863] 6. Plan Generation Phase

[0864] The server inputs feature data and sentiment data into a generative artificial intelligence system to generate a plan that best suits the user's needs and emotions.

[0865] 7. Plan Presentation Phase

[0866] The generative artificial intelligence generates a plan suggesting "three jogging sessions per week and a low-fat diet," and sends it to the terminal via the server.

[0867] 8. Dialogue and feedback phase based on user emotions

[0868] The device displays a plan to the user. The user provides feedback on the plan, such as, "Jogging is difficult. Are there any other exercises?"

[0869] The device sends this feedback to the server.

[0870] 9. Feedback Analysis and Plan Re-adjustment Phase

[0871] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate a plan under new conditions.

[0872] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[0873] The server resends the update plan to the device.

[0874] 10. Displaying updated plans

[0875] The device displays the updated plan to the user.

[0876] This invention allows users to receive more personalized plans according to their emotional state, achieving high user satisfaction by flexibly responding to the user's emotions.

[0877] The following describes the processing flow.

[0878] Step 1:

[0879] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[0880] Step 2:

[0881] The terminal receives user input information and sends it to the server.

[0882] Step 3:

[0883] The device uses a built-in emotion engine to determine the user's current emotional state. This emotion determination is performed in real time based on data such as the user's facial expressions and voice.

[0884] Step 4:

[0885] The device sends emotion data, determined by the emotion engine, to the server. This ensures that the user's input information and emotional state are received by the server.

[0886] Step 5:

[0887] The server analyzes the user's input and retrieves relevant information from the database. For example, it collects data such as past weight changes, dietary records, and exercise history.

[0888] Step 6:

[0889] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization.

[0890] Step 7:

[0891] The server uses machine learning models to extract key features from preprocessed data. For example, it identifies the user's past exercise patterns, dietary habits, and allergy information.

[0892] Step 8:

[0893] The server inputs the extracted features and sentiment data into a generative artificial intelligence system, instructing it to generate the optimal plan based on the user's needs and current emotional state.

[0894] Step 9:

[0895] Generative artificial intelligence analyzes input data and automatically generates the optimal plan for the user. For example, it might suggest a plan that includes jogging three times a week and a low-fat diet.

[0896] Step 10:

[0897] The server receives the generated plan, converts it to a predetermined format, and then sends it to the terminal.

[0898] Step 11:

[0899] The device displays the generated plan to the user. For example, it provides a detailed plan through smartphone notifications.

[0900] Step 12:

[0901] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[0902] Step 13:

[0903] The device receives user feedback and sends it to the server in real time.

[0904] Step 14:

[0905] The server analyzes user feedback and sentiment data, and, if necessary, instructs the generative artificial intelligence to generate plans under new conditions.

[0906] Step 15:

[0907] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might suggest "cycling three times a week instead."

[0908] Step 16:

[0909] The server converts the updated plan and sends it back to the terminal.

[0910] Step 17:

[0911] The device displays the updated plan to the user and provides the latest plan information.

[0912] Through these steps, the system responds quickly and accurately to user needs and emotions, generating and adjusting plans dynamically. This allows users to receive a personalized plan that is best suited to their situation and feelings.

[0913] (Example 2)

[0914] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0915] Traditional plan generation systems have struggled to provide personalized plans that take into account the user's emotional state. Therefore, there is a need for a system that generates flexible and optimal plans based on the user's current emotions and updates them in real time based on feedback.

[0916] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving existing information and sentiment information transmitted from the user and obtaining relevant information from a database, means for preprocessing the collected information and performing data cleaning and removal of outliers, and means for extracting features using the preprocessed data. This makes it possible to generate an optimal plan tailored to the user's sentiment state and update it in real time as needed.

[0917] A "user" refers to an individual who uses the system to input information and provides feedback on the proposed plan.

[0918] A "terminal" refers to an electronic device used by a user to input information, send the results of an analysis of their emotional state to a server, and receive a generated plan.

[0919] A "server" refers to a computer system that receives data sent by users, retrieves relevant information from a database, and performs data preprocessing, feature extraction, plan generation, and updates.

[0920] An "emotion engine" refers to an automated tool that analyzes a user's facial expressions and voice to obtain emotional information.

[0921] "Existing information" refers to past data and information about the current situation that users input into the system.

[0922] "Related information" refers to information that the server retrieves from the database, including user health data, past exercise history, and dietary records.

[0923] "Preprocessing" refers to the process of performing operations such as cleaning collected data, removing outliers, and supplementing incomplete data.

[0924] "Features" refer to important attributes or metrics extracted from pre-processed data.

[0925] A "generative artificial intelligence model" refers to a machine learning model that automatically generates the optimal plan based on user needs and emotional information.

[0926] "Plan" refers to exercise and dietary suggestions generated by generative artificial intelligence.

[0927] "Feedback" refers to the opinions and requests that users provide regarding the generated plan.

[0928] "Real-time" refers to a state where input and output occur almost simultaneously, with very little latency.

[0929] This invention relates to a system that combines existing information provided by the user with emotional information acquired by an emotion engine, automatically generates an optimal plan using a generative artificial intelligence model, and proposes it to the user. Below, we will describe the specific program processing of the system in natural language, and add specific examples.

[0930] System Configuration

[0931] This system consists of several main components, including the user, terminal, server, emotion engine, and generative artificial intelligence model. The roles and interactions of each component enable the system to automatically generate and present plans based on the user's needs and emotional state.

[0932] Hardware and software configuration

[0933] 1. Terminal:

[0934] These are devices that provide a user interface, such as smartphones, tablets, or PCs.

[0935] The built-in camera and microphone are used to analyze facial expressions and voice using an emotion engine.

[0936] The interface operates within a web browser or a specific application.

[0937] 2. Server:

[0938] It is a high-performance computer system that collects and preprocesses user data and generates plans using a generative artificial intelligence model.

[0939] It has a built-in database that stores and manages relevant information such as the user's health data, past exercise history, and dietary records.

[0940] 3. Emotional Engine:

[0941] This software uses computer vision and speech analysis technologies to analyze the user's emotional state.

[0942] For example, emotions can be identified from the user's facial expressions and voice tone.

[0943] 4. Generative artificial intelligence models:

[0944] For example, advanced generative artificial intelligence such as GPT-3 is used to analyze collected data and generate the optimal plan for the user.

[0945] Specific examples of actions

[0946] 1. User input phase:

[0947] The user enters their desired goals and requirements through the terminal's interface. For example, they might enter, "I want an exercise and meal plan to lose 5kg."

[0948] 2. Data Collection Phase:

[0949] The terminal receives information entered by the user and transmits this information to the server in real time.

[0950] 3. Emotional Data Collection Phase:

[0951] The device uses its built-in emotion engine to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[0952] 4. Data processing and preprocessing:

[0953] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database and performs preprocessing. The collected data is cleaned, outliers are removed, and the format is standardized.

[0954] 5. Feature extraction:

[0955] The server extracts features from the pre-processed data and identifies important attributes and metrics. For example, it uses machine learning algorithms to identify exercise patterns, dietary tendencies, allergy information, and so on.

[0956] 6. Plan generation:

[0957] The server inputs extracted features and sentiment data into a generative artificial intelligence model, instructing it to generate an optimal plan based on the user's needs and emotional state. The generative AI model analyzes this data and generates, for example, a "three-times-a-week-jogging and low-fat-meal plan."

[0958] 7. Presentation of the plan and feedback:

[0959] The generative artificial intelligence model sends the generated plan to the terminal via the server, and the terminal displays this plan to the user. The user then provides feedback on this plan.

[0960] The device sends user feedback to the server in real time, and the server analyzes the feedback and sentiment data.

[0961] 8. Readjust the plan:

[0962] The server sends the analysis results to a generative artificial intelligence model and instructs it to generate a plan under new conditions. For example, it might generate a "cycling plan three times a week instead."

[0963] Examples of prompt statements

[0964] For example, the following request is input to a generative artificial intelligence model:

[0965] "I'd like you to suggest an exercise and meal plan to lose 5kg. My current weight is 70kg, and my past exercise history mainly consists of jogging, but I don't have any specific dietary restrictions."

[0966] Through the above process, the present invention makes it possible to provide personalized plans according to the user's emotional state, thereby improving user satisfaction.

[0967] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0968] Step 1:

[0969] User input phase

[0970] The user enters their desired goals and requirements through the terminal's interface.

[0971] Specific input: The user enters, "I would like an exercise and meal plan to lose 5kg."

[0972] The device receives this information and stores it in its internal data structure.

[0973] Input: User's goals and requirements

[0974] Output: Saved user input data

[0975] Step 2:

[0976] Data collection phase

[0977] The terminal sends the information received from the user to the server.

[0978] Specific operation: The terminal sends saved user input data to the server in real time.

[0979] Input: User input data

[0980] Output: User data sent to the server

[0981] Step 3:

[0982] Emotional data collection phase

[0983] The device uses an emotion engine to analyze the user's facial expressions and voice to acquire emotion data.

[0984] Specific operation: Activates the facial recognition camera and microphone, processes the collected data with an emotion engine, and generates emotion data.

[0985] The device sends the acquired emotion data to the server.

[0986] Input: User's facial expression data and voice data

[0987] Output: Sentiment data sent to the server

[0988] Step 4:

[0989] Data acquisition and preprocessing phase

[0990] The server receives existing information and sentiment data sent by the user and retrieves relevant health data from the database.

[0991] Specific actions: Query the database to collect data such as past weight fluctuations, meal records, and exercise history.

[0992] The server preprocesses the collected data, cleaning it, removing outliers, and completing incomplete data.

[0993] Input: Existing user information and sentiment data

[0994] Output: Preprocessed data

[0995] Step 5:

[0996] Feature extraction phase

[0997] The server extracts features using the pre-processed data.

[0998] Specific operation: Machine learning algorithms (e.g., random forest, decision tree, etc.) are used to identify important attributes such as exercise patterns, dietary tendencies, and allergy information.

[0999] Input: Pre-processed data

[1000] Output: Extracted features

[1001] Step 6:

[1002] Plan generation phase

[1003] The server inputs the extracted feature vectors and sentiment data into a generative artificial intelligence model and instructs it to generate the optimal plan.

[1004] Specific operation: A generative artificial intelligence model (e.g., GPT-3) is input with prompt text and feature data to generate a plan.

[1005] Input: Extracted features and sentiment data

[1006] Output: Generated plan

[1007] Step 7:

[1008] Plan presentation phase

[1009] The generative artificial intelligence model sends the generated plan back to the server, which then formats the plan into a predetermined format.

[1010] Specific actions: Convert the generated plan to text format and attach additional information as needed.

[1011] The server sends the plan to the terminal.

[1012] Input: Generated plan

[1013] Output: Plan sent to the terminal

[1014] Step 8:

[1015] Dialogue and feedback phase based on user emotions

[1016] The device displays the plan generated for the user.

[1017] Specific actions: The plan will be displayed on the smartphone or PC screen, and the user will be notified.

[1018] Users provide feedback on the plan and enter it into their device. They might ask questions like, "Jogging is difficult. Are there any other exercises?"

[1019] The device sends this feedback to the server in real time.

[1020] Input: User feedback

[1021] Output: Feedback sent to the server

[1022] Step 9:

[1023] Feedback analysis and plan readjustment phase

[1024] The server analyzes user feedback and sentiment data.

[1025] Specific operation: Analyze feedback data and instruct the generative AI model to generate a plan under new conditions. Example: Generate a new plan that suggests cycling three times a week instead.

[1026] The generative artificial intelligence model generates a plan based on new conditions and sends it to the server.

[1027] Input: Feedback and sentiment data

[1028] Output: Updated plan

[1029] Step 10:

[1030] Display updated plans

[1031] The server sends the updated plan to the device.

[1032] The device displays the updated plan to the user.

[1033] Specific action: The updated plan will be displayed again on the smartphone or PC screen.

[1034] Input: Updated plan

[1035] Output: Plan displayed to the user

[1036] (Application Example 2)

[1037] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1038] Previously, the lack of a system to quickly provide personalized plans tailored to users' emotions and needs resulted in low user satisfaction. Furthermore, real-time product recommendations in physical stores were difficult, leading to an unoptimized shopping experience. To address these challenges, a system that understands users' emotional states and provides immediate, appropriate recommendations based on that understanding is essential.

[1039] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for analyzing the user's emotional state using an emotion engine and transmitting the emotional information to the server, and means for displaying optimal product suggestions to the user in the store through a smartphone or smart glasses interface. This enables real-time and personalized product suggestions that correspond to the user's emotional state.

[1040] "Existing information" refers to data and historical information that the user has provided in advance.

[1041] A "terminal" refers to a device used by a user to input or receive information through an interface. Examples include smartphones and smart glasses.

[1042] A "server" refers to a computer system that processes data sent by users and retrieves and manages related information.

[1043] A "database" refers to a system in which information is stored and managed.

[1044] "Preprocessing" refers to the process of cleansing and normalizing collected data and converting it into a format from which features can be extracted.

[1045] "Features" refer to important attributes and indicators extracted from data, which form the basis for analysis by machine learning models.

[1046] "Generative artificial intelligence" refers to machine learning and artificial intelligence technologies that automatically generate the optimal plan based on collected data and features.

[1047] A "plan" refers to a specific action plan or suggestion proposed by a generative artificial intelligence system based on the user's needs and emotions.

[1048] An "emotion engine" refers to a technology that analyzes a user's facial expressions and voice to determine their emotional state.

[1049] An "interface" refers to the screen or input device that a user uses to interact with a system.

[1050] "Feedback" refers to the reactions and opinions that users provide regarding the system.

[1051] A "smartphone" refers to a mobile communication device that has calling capabilities and a wide range of applications.

[1052] "Smart glasses" refer to a type of wearable device, specifically glasses with a display function, that presents information to the user.

[1053] "Product recommendations" refer to specific product information that is recommended based on the user's needs and emotional state.

[1054] A "physical store" refers to a store that has a physical presence and is a place where users can go in person to purchase products.

[1055] This invention relates to a system that provides personalized in-store product recommendations based on the user's emotional state. The specific configuration and operation of the system are described below.

[1056] Hardware and software to be used

[1057] hardware

[1058] 1. Device: A device used by the user (e.g., smartphone, smart glasses). This includes cameras and microphones.

[1059] 2. Server: A computer system that processes, manages, and runs generative artificial intelligence.

[1060] 3. Database: Stores user information, past purchase history, sentiment data, etc.

[1061] software

[1062] 1. Emotion Engine: A technology that analyzes a user's facial expressions and voice to determine their emotional state (e.g., FaceAPI, Microsoft Emotion API).

[1063] 2. Generative Artificial Intelligence: Machine learning and artificial intelligence technologies that automatically generate optimal plans based on collected data (e.g., GPT-4).

[1064] 3. Database management software: A system for operating and managing databases (e.g., MySQL).

[1065] 4. Machine learning frameworks: Used for data preprocessing, feature extraction, model building, and analysis (e.g., Scikit-learn, TensorFlow).

[1066] System operation

[1067] 1. User input

[1068] Users input their current shopping purpose and product categories of interest via their smartphone or smart glasses.

[1069] Specific example: The user enters "I want new summer clothes" into the device.

[1070] 2. Submitting the entered information

[1071] The terminal sends user input information to the server.

[1072] 3. Analysis of emotional state

[1073] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[1074] Specific example: Determining a user's level of excitement and interest based on their facial expressions and comments while viewing products in a store.

[1075] 4. Data Collection and Preprocessing

[1076] The server receives user input information and sentiment data, retrieves relevant data (such as past purchase history) from the database, and preprocesses that data. Preprocessing includes data cleansing, normalization, and consistency checks.

[1077] 5. Feature Extraction

[1078] The server extracts key features from pre-processed data using a machine learning model. These features represent the user's purchasing tendencies and current emotional state.

[1079] 6. Generating the optimal plan

[1080] The server inputs extracted feature data and sentiment data into a generative artificial intelligence system to generate an optimal product suggestion plan for the user. The generated plan is then converted by the server into a predetermined format and sent to the terminal.

[1081] Specific example: A plan is proposed that says, "New summer casual dresses are now 20% off."

[1082] 7. Plan provision and feedback

[1083] The terminal displays a proposed plan to the user, and the user provides feedback on it. The feedback is sent to the server, which updates the plan based on that feedback.

[1084] Specific example: A user enters feedback saying, "Is this dress available in a different color?"

[1085] Example of a prompt

[1086] "I want some new summer clothes."

[1087] "I'd like to know about any discounts on the products I've shown interest in."

[1088] "I'd like to check if this product is available in other colors."

[1089] These methods enable real-time and personalized product recommendations tailored to the user's emotional state, thereby increasing user satisfaction.

[1090] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1091] Step 1:

[1092] The user uses a device (smartphone or smart glasses) to input their current shopping purpose and product categories of interest. The user's request, "I want new summer clothes," is generated as input data.

[1093] Step 2:

[1094] The terminal sends user input information to the server. The terminal sends a request, which is the input data, and the server receives it. Data transmission occurs in real time.

[1095] Step 3:

[1096] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and generates emotion data. This emotion data includes emotional information (for example, excitement) determined from the user's facial expressions and statements when viewing a product.

[1097] Step 4:

[1098] The device sends the generated emotion data to the server. The emotion data is sent from the device to the server and used as input information for analysis.

[1099] Step 5:

[1100] The server receives user input information and sentiment data, and retrieves relevant information from the database. This relevant information includes data indicating the user's past purchase history, product preferences, and current sentiment state.

[1101] Step 6:

[1102] The server performs preprocessing on the collected information, removing outliers and normalizing the data. The preprocessed data is then formatted to ensure data consistency and allow for feature extraction.

[1103] Step 7:

[1104] The server inputs pre-processed data into a machine learning model and extracts features. These features include user purchasing tendencies and product categories of interest. Input data (pre-processed data) and output data (features) are generated at this stage.

[1105] Step 8:

[1106] The server inputs feature data and sentiment data into a generative artificial intelligence system to generate an optimal product recommendation plan. The generated plan is based on the user's needs and emotional state. For example, a plan such as "New summer casual dresses, now 20% off" might be generated.

[1107] Step 9:

[1108] The server converts the generated product proposal plan into a predetermined format and sends it to the terminal. The input data (generated plan) is converted into output data (plan in a predetermined format) and sent.

[1109] Step 10:

[1110] The device displays a suggested plan to the user. The user provides feedback on it (e.g., "Is this dress available in a different color?"). This feedback is then generated as new input data.

[1111] Step 11:

[1112] The device sends user feedback to the server in real time. Feedback data is sent and received by the server. This feedback data is then input as new conditions.

[1113] Step 12:

[1114] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate plans under new conditions. The generative AI generates new product suggestion plans that match the feedback and emotional state. For example, it might generate a product that suggests cycling three times a week instead.

[1115] Step 13:

[1116] The server converts the updated plan into the specified format and resends it to the terminal. The updated plan is then presented to the user.

[1117] Step 14:

[1118] The device displays updated product plans to the user. This allows the user to see new suggestions and have a more satisfying shopping experience.

[1119] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1120] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1121] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1122] [Third Embodiment]

[1123] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1124] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1125] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1126] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1127] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1128] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1129] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1130] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1131] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1132] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1133] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1134] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1135] This invention relates to a system that automatically generates an optimal plan using generative artificial intelligence based on the user's existing information and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[1136] In this system, the user, terminal, and server work together to generate a plan tailored to the user's needs. The following processes are performed in order to implement the invention.

[1137] Program processing

[1138] 1. User Input Phase

[1139] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input, "I want an exercise and meal plan to lose 5kg."

[1140] 2. Data Collection Phase

[1141] The terminal sends the user's input information to the server.

[1142] The server receives information sent by the user and retrieves related information from the database (e.g., past health data, purchase history, etc.).

[1143] 3. Data preprocessing and feature extraction phase

[1144] The server preprocesses the collected information. Preprocessing includes data cleaning, normalization, and removal of outliers.

[1145] The server extracts features from the pre-processed data by running it through a machine learning model. For example, these features might include "the user's weight fluctuation patterns and exercise preferences."

[1146] 4. Plan Generation Phase

[1147] The server inputs the extracted features into a generative artificial intelligence system and instructs it to generate the optimal plan for the user.

[1148] Generative artificial intelligence analyzes the input data and generates the optimal plan for the user. For example, it might recommend a plan that includes jogging three times a week and a low-fat diet.

[1149] 5. Plan Presentation Phase

[1150] The server converts the generated plan into a predetermined format and sends it to the terminal.

[1151] The device will display the plan to the user. For example, the user could view the detailed plan via a smartphone notification.

[1152] 6. Dialogue and Feedback Phase

[1153] Users ask questions and provide feedback about the generated plan. For example, they might say, "Jogging is difficult. Are there any other exercises?"

[1154] The device sends this feedback to the server.

[1155] The server analyzes the feedback and regenerates the plan as needed. Generative artificial intelligence generates a new plan based on the feedback, and the server sends it to the terminal. For example, it might suggest "cycling three times a week instead."

[1156] Specific example

[1157] Let's explain using a specific example of a user requesting a health plan.

[1158] 1. User input phase:

[1159] The user uses their device to input, "I want an exercise and meal plan to lose 5kg."

[1160] 2. Data Collection Phase:

[1161] The terminal sends this input to the server.

[1162] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database.

[1163] 3. Data preprocessing and feature extraction phase:

[1164] The server removes outliers from the collected data and normalizes the data.

[1165] The server uses machine learning models to extract features from the user's health data, such as weight fluctuation patterns and preferred exercise types.

[1166] 4. Plan generation phase:

[1167] The server inputs these features into a generative artificial intelligence system and instructs it to generate a plan that suits the user's needs.

[1168] The generative artificial intelligence generates a plan recommending "jogging three times a week and a low-fat diet."

[1169] 5. Plan presentation phase:

[1170] The server sends the generated plan to the terminal.

[1171] The device notifies the user and displays details within the app.

[1172] 6. Dialogue and Feedback Phase:

[1173] Users send feedback through their devices, saying things like, "Jogging is difficult. Are there any other exercises?"

[1174] The device sends this feedback to the server.

[1175] The server receives feedback and instructs the generative artificial intelligence to generate new movement options.

[1176] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[1177] The server sends the updated plan to the terminal and presents it to the user again.

[1178] As described above, the present invention allows users to receive plans based on their needs quickly and accurately. Furthermore, the plan can be dynamically adjusted through feedback, achieving high flexibility and user satisfaction.

[1179] The following describes the processing flow.

[1180] Step 1:

[1181] The user enters their existing information and specific requests through the terminal's interface. For example, they might enter a request such as "an exercise and meal plan to lose 5kg."

[1182] Step 2:

[1183] The terminal receives information entered by the user and sends that information to the server. This process takes place in real time.

[1184] Step 3:

[1185] After receiving user input, the server retrieves relevant detailed information from the database. For example, it collects data such as past weight fluctuations, dietary records, and exercise history.

[1186] Step 4:

[1187] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization. At this stage, the data is transformed into a consistent format.

[1188] Step 5:

[1189] The server extracts features using pre-processed data. Machine learning models are used to identify important attributes and metrics from the data. For example, it extracts user exercise patterns, dietary habits, and allergy information.

[1190] Step 6:

[1191] The server inputs the extracted features into the generative artificial intelligence and instructs it to generate a plan that suits the user's needs. The generative AI analyzes the input data and automatically generates the optimal plan for the user.

[1192] Step 7:

[1193] Generative artificial intelligence generates specific exercise and meal plans based on the analysis results. For example, it might suggest a plan for jogging three times a week and a low-fat diet.

[1194] Step 8:

[1195] The server receives the generated plan and converts it to a predetermined format before sending it to the user's terminal. This makes the plan easier for the user to understand.

[1196] Step 9:

[1197] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[1198] Step 10:

[1199] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[1200] Step 11:

[1201] The device sends user feedback to the server in real time.

[1202] Step 12:

[1203] The server receives user feedback and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[1204] Step 13:

[1205] Generative artificial intelligence incorporates user feedback to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[1206] Step 14:

[1207] The server receives the updated plan and sends it back to the device.

[1208] Step 15:

[1209] The device will display the plan to the user again and provide updated information.

[1210] Through these steps, the system responds quickly and accurately to user needs, generating and adjusting plans dynamically. This allows users to continuously receive individually customized plans.

[1211] (Example 1)

[1212] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1213] In modern society, health management and the provision of personalized plans are crucial issues. However, existing methods struggle to effectively handle large amounts of data and quickly generate personalized plans. Furthermore, they lack the functionality to dynamically adjust plans based on user feedback. This can lead to users not receiving the most suitable plan, resulting in decreased satisfaction.

[1214] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1215] In this invention, the server includes means for the user to input existing information, means for a terminal to receive existing information transmitted from the user and transfer it to the server, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the plan generated by the generative artificial intelligence into a predetermined format and transmit it to the terminal, means for the terminal to notify and display the plan to the user, means for the user to provide feedback and for the terminal to send the feedback to the server, and means for the server to update the plan based on the feedback. As a result, the user can obtain a personalized plan quickly and accurately, and since the plan is dynamically adjusted based on the feedback, user satisfaction can be increased.

[1216] A "user" refers to an individual who uses the system to request health management and plan generation.

[1217] A "server" refers to a series of computer devices that receive and process information sent by users.

[1218] "Terminal" refers to a device used by a user to input information and check the results (e.g., smartphone, tablet, personal computer).

[1219] "Existing information" refers to past health data and requests that users enter into the system.

[1220] A "database" refers to a collection of information that stores a user's past data and related information.

[1221] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected data, thereby converting it into a format that is easy to analyze.

[1222] "Features" refer to data that extracts specific characteristics or patterns that machine learning models use when generating plans.

[1223] "Generative artificial intelligence" refers to AI technology that generates the optimal plan for the user based on collected data.

[1224] A "plan" refers to specific action guidelines or suggestions created by a generative artificial intelligence system based on user needs.

[1225] "Feedback" refers to the opinions and requests for revisions that users provide regarding the generated plan.

[1226] "Update" refers to the process of generating new plans based on feedback or adjusting existing plans.

[1227] This invention relates to a system that automatically generates and proposes an optimal plan using generative artificial intelligence based on the user's existing information. The user, terminal, and server play important roles as the main components of this system. Specific embodiments are described below.

[1228] System Configuration

[1229] In this system, users input information using a device (e.g., smartphone, tablet, PC), and the device transmits that information to the server. The server receives the information, retrieves relevant information from the database, analyzes the data, and generates the optimal plan for the user.

[1230] Hardware and software to be used

[1231] User terminal: A device used by a user to input information. Specifically, this includes smartphones, tablets, and personal computers.

[1232] Server: A computer device that receives, preprocesses, analyzes, and sends generated plans based on data. Specific software options include Python, SQL, scikit-learn, pandas, and generative artificial intelligence (e.g., GPT-3).

[1233] Database: Data storage for storing existing user information and related information. SQL databases are often used.

[1234] Data processing and data calculation

[1235] 1. Receiving user input information

[1236] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input a specific request such as, "I want an exercise and meal plan to lose 5kg."

[1237] 2. Transfer and Reception of Information

[1238] The device sends information collected from the user to the server. This transmission is done via an HTTP request.

[1239] The server receives this request and retrieves relevant information (e.g., past health data or purchase history) from the database. Queries to the database are executed using the SQL language.

[1240] 3. Data preprocessing

[1241] The server preprocesses the collected data. Specific processing includes data cleaning (handling missing or outlier values) and data normalization (scaling). These processes are performed using libraries such as Python's pandas library.

[1242] 4. Feature extraction

[1243] The server inputs preprocessed data into a machine learning model and extracts features. The scikit-learn library is used for the specific processing. For example, features such as the user's weight fluctuation patterns and preferred exercise types might be extracted.

[1244] 5. Plan generation using generative artificial intelligence

[1245] The server inputs the extracted features as prompts into a generative artificial intelligence (AI) to generate the optimal plan. The generative AI (e.g., GPT-3) then performs analysis based on these prompts and generates a specific plan.

[1246] 6. Converting and sending the plan format.

[1247] The server converts the generated plan into a specified format (e.g., JSON format) and sends it to the terminal. This process is performed as an HTTP response.

[1248] The device receives this response and notifies the user. Specifically, it notifies the user via smartphone push notifications, etc., that "A new plan has been generated," and displays the plan details within the app.

[1249] Specific example

[1250] For example, by inputting prompt statements like the following into the AI ​​model, it is possible to generate a specific plan based on the user's request:

[1251] Example of a prompt:

[1252] "I want an exercise and meal plan to lose 5kg."

[1253] Based on this prompt, the AI ​​model generates text suggesting a "three-times-a-week-jogging and low-fat meal plan," and the server sends that text to the terminal for the user to receive.

[1254] The above describes an embodiment of the present invention, which allows users to quickly obtain individually customized plans and enables dynamic adjustments based on feedback.

[1255] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1256] Step 1:

[1257] User input phase

[1258] Input: Existing user information or specific requests. Example: "I want an exercise and meal plan to lose 5kg."

[1259] Output: Information entered by the user into the terminal

[1260] Specific operation: The user uses an application on their device (smartphone, computer, etc.) to enter their existing information (e.g., current weight, activity level, etc.) and specific requests into a text field, and then presses the "Send" button. This information is temporarily stored on the device.

[1261] Step 2:

[1262] Data collection phase

[1263] Input: User input information saved on the device

[1264] Output: Information to send to the server

[1265] Specific operation: The terminal packages the user's input information as an HTTP request and sends it to the server. The server receives this request and retrieves relevant information such as past health data and purchase history from the user database using SQL queries.

[1266] Step 3:

[1267] Data preprocessing and feature extraction phase

[1268] Input: User information collected by the server, related database information

[1269] Output: Preprocessed data and features

[1270] Specific operation: The server preprocesses the collected data. First, it cleans the data (e.g., removes outliers, imputes missing values), and then normalizes (scales) it. Next, it extracts features using a machine learning model (e.g., scikit-learn). These features include the user's weight change patterns and preferred exercise types.

[1271] Step 4:

[1272] Plan generation phase

[1273] Input: Preprocessed data and extracted features

[1274] Output: Generated plan

[1275] Specific operation: The server inputs the extracted features as prompts into a generative artificial intelligence model (e.g., GPT-3). The generative AI analyzes these prompts and generates a specific plan. For example, it might generate a plan for "jogging three times a week and a low-fat diet."

[1276] Step 5:

[1277] Plan presentation phase

[1278] Input: Generated plan

[1279] Output: Information presented to the user

[1280] Specific operation: The server converts the generated plan into a predetermined format (such as JSON format) and sends it to the device as an HTTP response. The device receives this response and notifies the user. Specifically, it uses smartphone push notifications to inform the user that "a new plan has been generated" and displays the detailed plan within the app.

[1281] Step 6:

[1282] Dialogue and Feedback Phase

[1283] Input: User feedback

[1284] Output: Updated plan

[1285] Specific operation: The user sends questions and feedback about the generated plan through their device. Specifically, they use an input form within the app to enter feedback such as, "Jogging is difficult. Are there any other exercises?" The device sends this feedback to the server as an HTTP request. The server receives the feedback and, if necessary, inputs new prompts into the generative AI to regenerate the plan. The server then sends the regenerated plan to the device and presents it to the user again. For example, it might suggest, "Cycling three times a week instead."

[1286] Through the above processing steps, users can quickly obtain individually customized plans, and the system allows for dynamic adjustments to those plans based on their feedback.

[1287] (Application Example 1)

[1288] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1289] Modern factories are required to maximize production efficiency while minimizing machine downtime. However, conventional methods lack the means to comprehensively analyze the operating data, inspection information, and current work status of each machine to generate optimal production schedules and work plans. As a result, problems such as inefficient production schedules and unplanned downtime occur. This invention aims to solve these problems and improve the production efficiency of factories.

[1290] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1291] In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and to obtain relevant information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for acquiring machine operation data, inspection information, and current work status in the factory and generating an optimal production schedule and work plan, and means for transmitting the generated production plan as instructions to the factory robots. This makes it possible to improve the overall operational efficiency of the factory and generate an optimal production schedule and work plan that minimizes downtime.

[1292] A "user" is a factory operator who operates the system and inputs the necessary information.

[1293] "Existing information" refers to information that users input into the system, such as product delivery dates, production quantities, and work instructions.

[1294] A "server" is a computer system that receives user information, retrieves related information from a database, and generates the optimal plan.

[1295] A "database" is a system that stores and manages various types of factory data, such as machine operation data, maintenance information, and work history.

[1296] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected information.

[1297] "Features" are important pieces of information extracted from pre-processed data to optimize production schedules.

[1298] "Generative artificial intelligence" refers to an artificial intelligence system that automatically generates the optimal plan based on the input data.

[1299] An "optimal plan" is a production schedule and work plan designed to maximize factory production efficiency and minimize downtime.

[1300] "Prescribed format" refers to the state in which the generated plan has been converted into a format that can be executed by a terminal or robot.

[1301] A "terminal" is a device used by users to input information or to review generated plans.

[1302] "Feedback" refers to requests for revisions or opinions that users provide regarding the generated plan.

[1303] "Machine operation data in a factory" refers to data on the operating status, usage time, and efficiency of machinery within a factory.

[1304] "Inspection information" refers to data on the maintenance history and maintenance plans of machinery within the factory.

[1305] "Current work status" refers to data on ongoing work and production activities within the factory.

[1306] A "production schedule" is a plan of time and work to efficiently manage and operate a factory's production process.

[1307] A "work plan" is a plan that outlines specific work instructions and procedures.

[1308] A "robot" is an automated machine that performs production activities and tasks within a factory.

[1309] This invention is a system that automatically generates and proposes an optimal production schedule and work plan to the user using generative artificial intelligence (generative AI model) based on factory machine operation data, inspection information, and current work status. The embodiments for carrying out this invention will be described in detail below.

[1310] System Configuration

[1311] This system consists of the following main components:

[1312] 1. Terminal as a user input device

[1313] 2. Servers that collect and process data

[1314] 3. Factory robots that perform specified tasks

[1315] Hardware and software used

[1316] Devices as terminals: Smartphones, tablets, personal computers, etc.

[1317] A powerful computer that acts as a server for data collection, preprocessing, feature extraction, and plan generation.

[1318] Machine learning software: The scikit-learn library in Python is used for data preprocessing, and RandomForestRegressor is used as the generative AI model.

[1319] Robot: An automated machine in a factory that performs a specified production activity.

[1320] Processing flow

[1321] 1. User input phase

[1322] The user inputs requirements such as delivery date and production quantity through the terminal interface. For example, a request might be, "I want to produce 1,000 units of the product by December 31st."

[1323] 2. Data Collection Phase

[1324] The server receives machine operation data, inspection information, and current work status collected within the factory.

[1325] 3. Data preprocessing and feature extraction phase

[1326] The server cleans, normalizes, and removes outliers from the collected data. Specifically, it uses the Python scikit-learn library.

[1327] After the data has been preprocessed, features are extracted. For example, these might include machine downtime or operating efficiency.

[1328] 4. Plan Generation Phase

[1329] The server inputs the extracted features into a generating AI model to automatically generate the optimal production schedule and work plan. Here, RandomForestRegressor is used to generate the plan.

[1330] The generated plan might be something like, "a production schedule that maximizes uptime and includes maintenance three times a week."

[1331] 5. Plan Presentation Phase

[1332] The generated plan is converted to a predetermined format on the server and sent to the terminal. The user reviews the plan through the terminal and sends instructions to the factory robots as needed.

[1333] 6. Dialogue and Feedback Phase

[1334] Users provide feedback on the generated plan. For example, they might say, "I'd like to extend the deadline to January 31st."

[1335] The server regenerates the plan based on this feedback and provides the updated plan to the user.

[1336] Specific example

[1337] 1. If the user is a factory operator

[1338] The operator enters a request stating, "We want to produce 1,000 units of product by December 31st."

[1339] The server collects factory machine operation data, inspection information, and current work status, extracts features, and then generates an optimal production schedule.

[1340] For example, a production schedule that maximizes operating time and includes maintenance three times a week is proposed.

[1341] If an operator provides feedback requesting an extension of the delivery date, the server analyzes the feedback and generates a new schedule.

[1342] Example of a prompt

[1343] For example, an operator might input the following information as an example of health data collection:

[1344] We collect the following information as user health data:

[1345] Weight changes over the past 6 months

[1346] Food diary

[1347] How many times a week do you exercise?

[1348] Please generate the optimal plan to lose 5kg.

[1349] Thus, the present invention can maximize factory production efficiency and eliminate inefficient production schedules and unplanned downtime.

[1350] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1351] Step 1:

[1352] The user inputs existing information through the terminal's interface. This input information includes production requests, such as "We want to produce 1000 units of product by December 31st." The input information is sent directly from the terminal to the server.

[1353] Step 2:

[1354] The server receives existing information sent from the terminal. It also retrieves relevant information from the database. This relevant information includes machine operation data, inspection information, and current work status in the factory. The server collects all of this data together.

[1355] Step 3:

[1356] The server preprocesses the collected information. Preprocessing involves cleaning, normalizing, and removing outliers from the data. Specifically, it uses the Python scikit-learn library to scale and filter the data to create an accurate dataset.

[1357] Step 4:

[1358] The server extracts features from the pre-processed data. These features include, for example, the average machine downtime and operating efficiency. These features are important indicators for optimizing production schedules.

[1359] Step 5:

[1360] The server inputs the extracted features into a generating AI model (RandomForestRegressor) to generate the optimal production schedule and work plan. For example, a production schedule that maximizes uptime and includes three maintenance cycles per week is generated. The generated production plan is temporarily stored on the server.

[1361] Step 6:

[1362] The generated plan is converted to a predetermined format by the server. The converted plan is sent to the terminal and displayed on the user's (factory operator's) screen. The user can review the production plan and modify the work instructions as needed.

[1363] Step 7:

[1364] Users provide feedback through their devices. This feedback may include requests such as, "I would like to extend the delivery date to January 31st." The feedback is sent from the device to the server.

[1365] Step 8:

[1366] The server updates the plan based on the feedback received. The generating AI model analyzes the new feedback information and regenerates the production schedule as needed. For example, a "new production schedule that accommodates the extended delivery date" is generated.

[1367] Step 9:

[1368] The updated plan is resent from the server to the terminal and displayed again to the user. The user reviews the new plan and sends it to the robot as the final work instruction. Once this is complete, the system begins processing the next cycle.

[1369] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1370] This invention relates to a system that combines existing information provided by the user with emotional information read by an emotion engine, automatically generates an optimal plan using generative artificial intelligence, and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[1371] In this system, the user, terminal, server, and emotion engine work together to generate a plan that is tailored to the user's needs and emotional state. The following processes are performed in an embodiment of the invention.

[1372] Program processing

[1373] 1. User Input Phase

[1374] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[1375] 2. Data Collection Phase

[1376] The terminal receives information entered by the user and sends it to the server. This process takes place in real time.

[1377] 3. Emotional Data Collection Phase

[1378] The device analyzes the user's emotional state using an emotion engine and sends the results to the server. The emotion engine uses facial recognition and voice analysis technologies to determine the emotions the user is currently feeling.

[1379] 4. Data Acquisition and Preprocessing Phase

[1380] The server receives user input information and sentiment data, and retrieves relevant information from the database. For example, it collects past weight fluctuations, food records, and exercise history.

[1381] The server preprocesses the collected information, cleaning the data, removing outliers, and completing incomplete data. At this stage, the data is converted into a consistent format.

[1382] 5. Feature Extraction Phase

[1383] The server extracts features using pre-processed data. Machine learning models are used to identify important attributes and metrics from the data. For example, it might extract user exercise patterns, dietary habits, and allergy information.

[1384] 6. Plan Generation Phase

[1385] The server inputs the extracted features and sentiment data into a generative artificial intelligence system and instructs it to generate a plan that is suitable for the user's needs and current emotional state.

[1386] Generative artificial intelligence analyzes the input data and automatically generates the optimal plan for the user.

[1387] 7. Plan Presentation Phase

[1388] Generative artificial intelligence generates specific exercise and meal plans based on the analysis results. For example, it might suggest a plan for jogging three times a week and a low-fat diet.

[1389] The server converts the generated plan and formats it into the required format before sending it to the user's terminal.

[1390] 8. Dialogue and feedback phase based on user emotions

[1391] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[1392] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[1393] The device sends this feedback to the server in real time.

[1394] 9. Feedback Analysis and Plan Re-adjustment Phase

[1395] The server receives user feedback and emotional data determined by the emotion engine, and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[1396] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[1397] The server converts the updated plan and sends it back to the terminal.

[1398] 10. Displaying updated plans

[1399] The device will display the plan to the user again and provide updated information.

[1400] Specific example

[1401] The following process is a concrete example of what happens when a user requests a health plan.

[1402] 1. User Input Phase

[1403] The user uses their device to input, "I want an exercise and meal plan to lose 5kg."

[1404] 2. Data Collection Phase

[1405] The terminal sends this input to the server.

[1406] 3. Emotional Data Collection Phase

[1407] The device uses an emotion engine to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[1408] 4. Data Acquisition and Preprocessing Phase

[1409] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database.

[1410] 5. Feature Extraction Phase

[1411] The server removes outliers from the collected data and normalizes the data. Next, it extracts features.

[1412] 6. Plan Generation Phase

[1413] The server inputs feature data and sentiment data into a generative artificial intelligence system to generate a plan that best suits the user's needs and emotions.

[1414] 7. Plan Presentation Phase

[1415] The generative artificial intelligence generates a plan suggesting "three jogging sessions per week and a low-fat diet," and sends it to the terminal via the server.

[1416] 8. Dialogue and feedback phase based on user emotions

[1417] The device displays a plan to the user. The user provides feedback on the plan, such as, "Jogging is difficult. Are there any other exercises?"

[1418] The device sends this feedback to the server.

[1419] 9. Feedback Analysis and Plan Re-adjustment Phase

[1420] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate a plan under new conditions.

[1421] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[1422] The server resends the update plan to the device.

[1423] 10. Displaying updated plans

[1424] The device displays the updated plan to the user.

[1425] This invention allows users to receive more personalized plans according to their emotional state, achieving high user satisfaction by flexibly responding to the user's emotions.

[1426] The following describes the processing flow.

[1427] Step 1:

[1428] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[1429] Step 2:

[1430] The terminal receives user input information and sends it to the server.

[1431] Step 3:

[1432] The device uses a built-in emotion engine to determine the user's current emotional state. This emotion determination is performed in real time based on data such as the user's facial expressions and voice.

[1433] Step 4:

[1434] The device sends emotion data, determined by the emotion engine, to the server. This ensures that the user's input information and emotional state are received by the server.

[1435] Step 5:

[1436] The server analyzes the user's input and retrieves relevant information from the database. For example, it collects data such as past weight changes, dietary records, and exercise history.

[1437] Step 6:

[1438] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization.

[1439] Step 7:

[1440] The server uses machine learning models to extract key features from pre-processed data. For example, it identifies the user's past exercise patterns, dietary habits, and allergy information.

[1441] Step 8:

[1442] The server inputs the extracted features and sentiment data into a generative artificial intelligence system, instructing it to generate the optimal plan based on the user's needs and current emotional state.

[1443] Step 9:

[1444] Generative artificial intelligence analyzes input data and automatically generates the optimal plan for the user. For example, it might suggest a plan that includes jogging three times a week and a low-fat diet.

[1445] Step 10:

[1446] The server receives the generated plan, converts it to a predetermined format, and then sends it to the terminal.

[1447] Step 11:

[1448] The device displays the generated plan to the user. For example, it provides a detailed plan through smartphone notifications.

[1449] Step 12:

[1450] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[1451] Step 13:

[1452] The device receives user feedback and sends it to the server in real time.

[1453] Step 14:

[1454] The server analyzes user feedback and sentiment data, and, if necessary, instructs the generative artificial intelligence to generate plans under new conditions.

[1455] Step 15:

[1456] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might suggest "cycling three times a week instead."

[1457] Step 16:

[1458] The server converts the updated plan and sends it back to the terminal.

[1459] Step 17:

[1460] The device displays the updated plan to the user and provides the latest plan information.

[1461] Through these steps, the system responds quickly and accurately to user needs and emotions, generating and adjusting plans dynamically. This allows users to receive a personalized plan that is best suited to their situation and feelings.

[1462] (Example 2)

[1463] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1464] Traditional plan generation systems have struggled to provide personalized plans that take into account the user's emotional state. Therefore, there is a need for a system that generates flexible and optimal plans based on the user's current emotions and updates them in real time based on feedback.

[1465] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving existing information and sentiment information transmitted from the user and obtaining relevant information from a database, means for preprocessing the collected information and performing data cleaning and removal of outliers, and means for extracting features using the preprocessed data. This makes it possible to generate an optimal plan tailored to the user's sentiment state and update it in real time as needed.

[1466] A "user" refers to an individual who uses the system to input information and provides feedback on the proposed plan.

[1467] A "terminal" refers to an electronic device used by a user to input information, send the results of an analysis of their emotional state to a server, and receive a generated plan.

[1468] A "server" refers to a computer system that receives data sent by users, retrieves relevant information from a database, and performs data preprocessing, feature extraction, plan generation, and updates.

[1469] An "emotion engine" refers to an automated tool that analyzes a user's facial expressions and voice to obtain emotional information.

[1470] "Existing information" refers to past data and information about the current situation that users input into the system.

[1471] "Related information" refers to information that the server retrieves from the database, including user health data, past exercise history, and dietary records.

[1472] "Preprocessing" refers to the process of performing operations such as cleaning collected data, removing outliers, and supplementing incomplete data.

[1473] "Features" refer to important attributes or metrics extracted from pre-processed data.

[1474] A "generative artificial intelligence model" refers to a machine learning model that automatically generates the optimal plan based on user needs and emotional information.

[1475] "Plan" refers to exercise and dietary suggestions generated by generative artificial intelligence.

[1476] "Feedback" refers to the opinions and requests that users provide regarding the generated plan.

[1477] "Real-time" refers to a state where input and output occur almost simultaneously, with very little latency.

[1478] This invention relates to a system that combines existing information provided by the user with emotional information acquired by an emotion engine, automatically generates an optimal plan using a generative artificial intelligence model, and proposes it to the user. Below, we will describe the specific program processing of the system in natural language, and add specific examples.

[1479] System Configuration

[1480] This system consists of several main components, including the user, terminal, server, emotion engine, and generative artificial intelligence model. The roles and interactions of each component enable the system to automatically generate and present plans based on the user's needs and emotional state.

[1481] Hardware and software configuration

[1482] 1. Terminal:

[1483] These are devices that provide a user interface, such as smartphones, tablets, or PCs.

[1484] The built-in camera and microphone are used to analyze facial expressions and voice using an emotion engine.

[1485] The interface operates within a web browser or a specific application.

[1486] 2. Server:

[1487] It is a high-performance computer system that collects and preprocesses user data and generates plans using a generative artificial intelligence model.

[1488] It has a built-in database that stores and manages relevant information such as the user's health data, past exercise history, and dietary records.

[1489] 3. Emotional Engine:

[1490] This software uses computer vision and speech analysis technologies to analyze the user's emotional state.

[1491] For example, emotions can be identified from the user's facial expressions and voice tone.

[1492] 4. Generative artificial intelligence models:

[1493] For example, advanced generative artificial intelligence such as GPT-3 is used to analyze collected data and generate the optimal plan for the user.

[1494] Specific examples of actions

[1495] 1. User input phase:

[1496] The user enters their desired goals and requirements through the terminal's interface. For example, they might enter, "I want an exercise and meal plan to lose 5kg."

[1497] 2. Data Collection Phase:

[1498] The terminal receives information entered by the user and transmits this information to the server in real time.

[1499] 3. Emotional Data Collection Phase:

[1500] The device uses its built-in emotion engine to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[1501] 4. Data processing and preprocessing:

[1502] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database and performs preprocessing. The collected data is cleaned, outliers are removed, and the format is standardized.

[1503] 5. Feature extraction:

[1504] The server extracts features from the pre-processed data and identifies important attributes and metrics. For example, it uses machine learning algorithms to identify exercise patterns, dietary tendencies, allergy information, and so on.

[1505] 6. Plan generation:

[1506] The server inputs extracted features and sentiment data into a generative artificial intelligence model, instructing it to generate an optimal plan based on the user's needs and emotional state. The generative AI model analyzes this data and generates, for example, a "three-times-a-week-jogging and low-fat-meal plan."

[1507] 7. Presentation of the plan and feedback:

[1508] The generative artificial intelligence model sends the generated plan to the terminal via the server, and the terminal displays this plan to the user. The user then provides feedback on this plan.

[1509] The device sends user feedback to the server in real time, and the server analyzes the feedback and sentiment data.

[1510] 8. Readjust the plan:

[1511] The server sends the analysis results to a generative artificial intelligence model and instructs it to generate a plan under new conditions. For example, it might generate a "cycling plan three times a week instead."

[1512] Examples of prompt statements

[1513] For example, the following request is input to a generative artificial intelligence model:

[1514] "I'd like you to suggest an exercise and meal plan to lose 5kg. My current weight is 70kg, and my past exercise history mainly consists of jogging, but I don't have any specific dietary restrictions."

[1515] Through the above process, the present invention makes it possible to provide personalized plans according to the user's emotional state, thereby improving user satisfaction.

[1516] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1517] Step 1:

[1518] User input phase

[1519] The user enters their desired goals and requirements through the terminal's interface.

[1520] Specific input: The user enters, "I would like an exercise and meal plan to lose 5kg."

[1521] The device receives this information and stores it in its internal data structure.

[1522] Input: User's goals and requirements

[1523] Output: Saved user input data

[1524] Step 2:

[1525] Data collection phase

[1526] The terminal sends the information received from the user to the server.

[1527] Specific operation: The terminal sends saved user input data to the server in real time.

[1528] Input: User input data

[1529] Output: User data sent to the server

[1530] Step 3:

[1531] Emotional data collection phase

[1532] The device uses an emotion engine to analyze the user's facial expressions and voice to acquire emotion data.

[1533] Specific operation: Activates the facial recognition camera and microphone, processes the collected data with an emotion engine, and generates emotion data.

[1534] The device sends the acquired emotion data to the server.

[1535] Input: User's facial expression data and voice data

[1536] Output: Sentiment data sent to the server

[1537] Step 4:

[1538] Data acquisition and preprocessing phase

[1539] The server receives existing information and sentiment data sent by the user and retrieves relevant health data from the database.

[1540] Specific actions: Query the database to collect data such as past weight fluctuations, meal records, and exercise history.

[1541] The server preprocesses the collected data, cleaning it, removing outliers, and completing incomplete data.

[1542] Input: Existing user information and sentiment data

[1543] Output: Preprocessed data

[1544] Step 5:

[1545] Feature extraction phase

[1546] The server extracts features using the pre-processed data.

[1547] Specific operation: Machine learning algorithms (e.g., random forest, decision tree, etc.) are used to identify important attributes such as exercise patterns, dietary tendencies, and allergy information.

[1548] Input: Pre-processed data

[1549] Output: Extracted features

[1550] Step 6:

[1551] Plan generation phase

[1552] The server inputs the extracted feature vectors and sentiment data into a generative artificial intelligence model and instructs it to generate the optimal plan.

[1553] Specific operation: A generative artificial intelligence model (e.g., GPT-3) is input with prompt text and feature data to generate a plan.

[1554] Input: Extracted features and sentiment data

[1555] Output: Generated plan

[1556] Step 7:

[1557] Plan presentation phase

[1558] The generative artificial intelligence model sends the generated plan back to the server, which then formats the plan into a predetermined format.

[1559] Specific actions: Convert the generated plan to text format and attach additional information as needed.

[1560] The server sends the plan to the terminal.

[1561] Input: Generated plan

[1562] Output: Plan sent to the terminal

[1563] Step 8:

[1564] Dialogue and feedback phase based on user emotions

[1565] The device displays the plan generated for the user.

[1566] Specific actions: The plan will be displayed on the smartphone or PC screen, and the user will be notified.

[1567] Users provide feedback on the plan and enter it into their device. They might ask questions like, "Jogging is difficult. Are there any other exercises?"

[1568] The device sends this feedback to the server in real time.

[1569] Input: User feedback

[1570] Output: Feedback sent to the server

[1571] Step 9:

[1572] Feedback analysis and plan readjustment phase

[1573] The server analyzes user feedback and sentiment data.

[1574] Specific operation: Analyze feedback data and instruct the generative AI model to generate a plan under new conditions. Example: Generate a new plan that suggests cycling three times a week instead.

[1575] The generative artificial intelligence model generates a plan based on new conditions and sends it to the server.

[1576] Input: Feedback and sentiment data

[1577] Output: Updated plan

[1578] Step 10:

[1579] Displaying updated plans

[1580] The server sends the updated plan to the device.

[1581] The device displays the updated plan to the user.

[1582] Specific action: The updated plan will be displayed again on the smartphone or PC screen.

[1583] Input: Updated plan

[1584] Output: Plan displayed to the user

[1585] (Application Example 2)

[1586] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1587] Previously, the lack of a system to quickly provide personalized plans tailored to users' emotions and needs resulted in low user satisfaction. Furthermore, providing appropriate product recommendations in real time at physical stores was difficult, resulting in an unoptimized user shopping experience. To address these challenges, a system that understands users' emotional states and provides immediate, appropriate recommendations based on that understanding is essential.

[1588] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for analyzing the user's emotional state using an emotion engine and transmitting the emotional information to the server, and means for displaying optimal product suggestions to the user in the store through a smartphone or smart glasses interface. This enables real-time and personalized product suggestions that correspond to the user's emotional state.

[1589] "Existing information" refers to data and historical information that the user has provided in advance.

[1590] A "terminal" refers to a device that a user uses to input or receive information through an interface. Examples include smartphones and smart glasses.

[1591] A "server" refers to a computer system that processes data sent by users and retrieves and manages related information.

[1592] A "database" refers to a system in which information is stored and managed.

[1593] "Preprocessing" refers to the process of cleansing and normalizing collected data and converting it into a format from which features can be extracted.

[1594] "Features" refer to important attributes and indicators extracted from data, and the underlying data that is analyzed by machine learning models.

[1595] "Generative artificial intelligence" refers to machine learning and artificial intelligence technologies that automatically generate the optimal plan based on collected data and features.

[1596] A "plan" refers to a specific action plan or suggestion proposed by a generative artificial intelligence system based on the user's needs and emotions.

[1597] An "emotion engine" refers to a technology that analyzes a user's facial expressions and voice to determine their emotional state.

[1598] An "interface" refers to the screen or input device that a user uses to interact with a system.

[1599] "Feedback" refers to the reactions and opinions that users provide regarding the system.

[1600] A "smartphone" refers to a mobile communication device that has calling capabilities and a wide range of applications.

[1601] "Smart glasses" refer to a type of wearable device, specifically glasses with a display function, that presents information to the user.

[1602] "Product recommendations" refer to specific product information that is recommended based on the user's needs and emotional state.

[1603] A "physical store" refers to a store that has a physical presence and is a place where users can go in person to purchase products.

[1604] This invention relates to a system that provides personalized in-store product recommendations based on the user's emotional state. The specific configuration and operation of the system are described below.

[1605] Hardware and software to be used

[1606] hardware

[1607] 1. Device: A device used by the user (e.g., smartphone, smart glasses). This includes cameras and microphones.

[1608] 2. Server: A computer system that processes, manages, and runs generative artificial intelligence.

[1609] 3. Database: Stores user information, past purchase history, sentiment data, etc.

[1610] software

[1611] 1. Emotion Engine: A technology that analyzes a user's facial expressions and voice to determine their emotional state (e.g., FaceAPI, Microsoft Emotion API).

[1612] 2. Generative Artificial Intelligence: Machine learning and artificial intelligence technologies that automatically generate optimal plans based on collected data (e.g., GPT-4).

[1613] 3. Database management software: A system for operating and managing databases (e.g., MySQL).

[1614] 4. Machine learning frameworks: Used for data preprocessing, feature extraction, model building, and analysis (e.g., Scikit-learn, TensorFlow).

[1615] System operation

[1616] 1. User input

[1617] Users input their current shopping purpose and product categories of interest via their smartphone or smart glasses.

[1618] Specific example: The user enters "I want new summer clothes" into the device.

[1619] 2. Submitting the entered information

[1620] The terminal sends user input information to the server.

[1621] 3. Analysis of emotional state

[1622] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[1623] Specific example: Determining a user's level of excitement and interest based on their facial expressions and comments while viewing products in a store.

[1624] 4. Data Collection and Preprocessing

[1625] The server receives user input information and sentiment data, retrieves relevant data (such as past purchase history) from the database, and preprocesses that data. Preprocessing includes data cleansing, normalization, and consistency checks.

[1626] 5. Feature Extraction

[1627] The server extracts key features from pre-processed data using a machine learning model. These features represent the user's purchasing tendencies and current emotional state.

[1628] 6. Generating the optimal plan

[1629] The server inputs extracted feature vectors and sentiment data into a generative artificial intelligence system to generate an optimal product suggestion plan for the user. The generated plan is then converted by the server into a predetermined format and sent to the terminal.

[1630] Specific example: A plan is proposed that says, "New summer casual dresses are now 20% off."

[1631] 7. Plan provision and feedback

[1632] The terminal displays a proposed plan to the user, and the user provides feedback on it. The feedback is sent to the server, which updates the plan based on that feedback.

[1633] Specific example: A user enters feedback saying, "Is this dress available in a different color?"

[1634] Example of a prompt

[1635] "I want some new summer clothes."

[1636] "I'd like to know about any discounts on the products I've shown interest in."

[1637] "I'd like to check if this product is available in other colors."

[1638] These methods enable real-time and personalized product recommendations tailored to the user's emotional state, thereby increasing user satisfaction.

[1639] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1640] Step 1:

[1641] The user uses a device (smartphone or smart glasses) to input their current shopping purpose and product categories of interest. The user's request, "I want new summer clothes," is generated as input data.

[1642] Step 2:

[1643] The terminal sends user input information to the server. The terminal sends a request, which is the input data, and the server receives it. Data transmission occurs in real time.

[1644] Step 3:

[1645] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and generates emotion data. This emotion data includes emotional information (for example, excitement) determined from the user's facial expressions and statements when viewing a product.

[1646] Step 4:

[1647] The device sends the generated emotion data to the server. The emotion data is sent from the device to the server and used as input information for analysis.

[1648] Step 5:

[1649] The server receives user input information and sentiment data, and retrieves relevant information from the database. This relevant information includes data indicating the user's past purchase history, product preferences, and current sentiment state.

[1650] Step 6:

[1651] The server performs preprocessing on the collected information, removing outliers and normalizing the data. The preprocessed data is then formatted to ensure data consistency and allow for feature extraction.

[1652] Step 7:

[1653] The server inputs pre-processed data into a machine learning model and extracts features. These features include user purchasing tendencies and product categories of interest. Input data (pre-processed data) and output data (features) are generated at this stage.

[1654] Step 8:

[1655] The server inputs feature data and sentiment data into a generative artificial intelligence system to generate an optimal product recommendation plan. The generated plan is based on the user's needs and emotional state. For example, a plan such as "New summer casual dresses, now 20% off" might be generated.

[1656] Step 9:

[1657] The server converts the generated product proposal plan into a predetermined format and sends it to the terminal. The input data (generated plan) is converted into output data (plan in a predetermined format) and sent.

[1658] Step 10:

[1659] The device displays a suggested plan to the user. The user provides feedback on it (e.g., "Is this dress available in a different color?"). This feedback is then generated as new input data.

[1660] Step 11:

[1661] The device sends user feedback to the server in real time. Feedback data is sent and received by the server. This feedback data is then input as new conditions.

[1662] Step 12:

[1663] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate plans under new conditions. The generative AI generates new product suggestion plans that match the feedback and emotional state. For example, it might generate a product that suggests cycling three times a week instead.

[1664] Step 13:

[1665] The server converts the updated plan into the specified format and resends it to the terminal. The updated plan is then presented to the user.

[1666] Step 14:

[1667] The device displays updated product plans to the user. This allows the user to see new suggestions and have a more satisfying shopping experience.

[1668] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1669] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1670] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1671] [Fourth Embodiment]

[1672] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1673] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1674] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1675] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1676] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1677] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1678] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1679] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1680] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1681] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1682] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1683] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1684] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1685] This invention relates to a system that automatically generates an optimal plan using generative artificial intelligence based on the user's existing information and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[1686] In this system, the user, terminal, and server work together to generate a plan tailored to the user's needs. The following processes are performed in order to implement the invention.

[1687] Program processing

[1688] 1. User Input Phase

[1689] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input, "I want an exercise and meal plan to lose 5kg."

[1690] 2. Data Collection Phase

[1691] The terminal sends the user's input information to the server.

[1692] The server receives information sent by the user and retrieves related information from the database (e.g., past health data, purchase history, etc.).

[1693] 3. Data preprocessing and feature extraction phase

[1694] The server preprocesses the collected information. Preprocessing includes data cleaning, normalization, and removal of outliers.

[1695] The server extracts features from the pre-processed data by running it through a machine learning model. For example, these features might include "the user's weight fluctuation patterns and exercise preferences."

[1696] 4. Plan Generation Phase

[1697] The server inputs the extracted features into a generative artificial intelligence system and instructs it to generate the optimal plan for the user.

[1698] Generative artificial intelligence analyzes the input data and generates the optimal plan for the user. For example, it might recommend a plan that includes jogging three times a week and a low-fat diet.

[1699] 5. Plan Presentation Phase

[1700] The server converts the generated plan into a predetermined format and sends it to the terminal.

[1701] The device will display the plan to the user. For example, the user could view the detailed plan via a smartphone notification.

[1702] 6. Dialogue and Feedback Phase

[1703] Users ask questions and provide feedback about the generated plan. For example, they might say, "Jogging is difficult. Are there any other exercises?"

[1704] The device sends this feedback to the server.

[1705] The server analyzes the feedback and regenerates the plan as needed. Generative artificial intelligence generates a new plan based on the feedback, and the server sends it to the terminal. For example, it might suggest "cycling three times a week instead."

[1706] Specific example

[1707] Let's explain using a specific example of a user requesting a health plan.

[1708] 1. User input phase:

[1709] The user uses their device to input, "I want an exercise and meal plan to lose 5kg."

[1710] 2. Data Collection Phase:

[1711] The terminal sends this input to the server.

[1712] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database.

[1713] 3. Data preprocessing and feature extraction phase:

[1714] The server removes outliers from the collected data and normalizes the data.

[1715] The server uses machine learning models to extract features from the user's health data, such as weight fluctuation patterns and preferred exercise types.

[1716] 4. Plan generation phase:

[1717] The server inputs these features into a generative artificial intelligence system and instructs it to generate a plan that suits the user's needs.

[1718] The generative artificial intelligence generates a plan recommending "jogging three times a week and a low-fat diet."

[1719] 5. Plan presentation phase:

[1720] The server sends the generated plan to the terminal.

[1721] The device notifies the user and displays details within the app.

[1722] 6. Dialogue and Feedback Phase:

[1723] Users send feedback through their devices, saying things like, "Jogging is difficult. Are there any other exercises?"

[1724] The device sends this feedback to the server.

[1725] The server receives feedback and instructs the generative artificial intelligence to generate new movement options.

[1726] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[1727] The server sends the updated plan to the terminal and presents it to the user again.

[1728] As described above, the present invention allows users to receive plans based on their needs quickly and accurately. Furthermore, the plan can be dynamically adjusted through feedback, achieving high flexibility and user satisfaction.

[1729] The following describes the processing flow.

[1730] Step 1:

[1731] The user enters their existing information and specific requests through the terminal's interface. For example, they might enter a request such as "an exercise and meal plan to lose 5kg."

[1732] Step 2:

[1733] The terminal receives information entered by the user and sends that information to the server. This process takes place in real time.

[1734] Step 3:

[1735] After receiving user input, the server retrieves relevant detailed information from the database. For example, it collects data such as past weight fluctuations, dietary records, and exercise history.

[1736] Step 4:

[1737] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization. At this stage, the data is transformed into a consistent format.

[1738] Step 5:

[1739] The server extracts features using pre-processed data. Machine learning models are used to identify important attributes and metrics from the data. For example, it extracts user exercise patterns, dietary habits, and allergy information.

[1740] Step 6:

[1741] The server inputs the extracted features into the generative artificial intelligence and instructs it to generate a plan that suits the user's needs. The generative AI analyzes the input data and automatically generates the optimal plan for the user.

[1742] Step 7:

[1743] Generative artificial intelligence generates specific exercise and meal plans based on the analysis results. For example, it might suggest a plan for jogging three times a week and a low-fat diet.

[1744] Step 8:

[1745] The server receives the generated plan and converts it to a predetermined format before sending it to the user's terminal. This makes the plan easier for the user to understand.

[1746] Step 9:

[1747] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[1748] Step 10:

[1749] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[1750] Step 11:

[1751] The device sends user feedback to the server in real time.

[1752] Step 12:

[1753] The server receives user feedback and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[1754] Step 13:

[1755] Generative artificial intelligence incorporates user feedback to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[1756] Step 14:

[1757] The server receives the updated plan and sends it back to the device.

[1758] Step 15:

[1759] The device will display the plan to the user again and provide updated information.

[1760] Through these steps, the system responds quickly and accurately to user needs, generating and adjusting plans dynamically. This allows users to continuously receive individually customized plans.

[1761] (Example 1)

[1762] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1763] In modern society, health management and the provision of personalized plans are crucial issues. However, existing methods struggle to effectively handle large amounts of data and quickly generate personalized plans. Furthermore, they lack the functionality to dynamically adjust plans based on user feedback. This can lead to users not receiving the most suitable plan, resulting in decreased satisfaction.

[1764] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1765] In this invention, the server includes means for the user to input existing information, means for a terminal to receive existing information transmitted from the user and transfer it to the server, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the plan generated by the generative artificial intelligence into a predetermined format and transmit it to the terminal, means for the terminal to notify and display the plan to the user, means for the user to provide feedback and for the terminal to send the feedback to the server, and means for the server to update the plan based on the feedback. As a result, the user can obtain a personalized plan quickly and accurately, and since the plan is dynamically adjusted based on the feedback, user satisfaction can be increased.

[1766] A "user" refers to an individual who uses the system to request health management and plan generation.

[1767] A "server" refers to a series of computer devices that receive and process information sent by users.

[1768] "Terminal" refers to a device used by a user to input information and check the results (e.g., smartphone, tablet, personal computer).

[1769] "Existing information" refers to past health data and requests that users enter into the system.

[1770] A "database" refers to a collection of information that stores a user's past data and related information.

[1771] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected data, thereby converting it into a format that is easy to analyze.

[1772] "Features" refer to data that extracts specific characteristics or patterns that machine learning models use when generating plans.

[1773] "Generative artificial intelligence" refers to AI technology that generates the optimal plan for the user based on collected data.

[1774] A "plan" refers to specific action guidelines or suggestions created by a generative artificial intelligence system based on user needs.

[1775] "Feedback" refers to the opinions and requests for revisions that users provide regarding the generated plan.

[1776] "Update" refers to the process of generating new plans based on feedback or adjusting existing plans.

[1777] This invention relates to a system that automatically generates and proposes an optimal plan using generative artificial intelligence based on the user's existing information. The user, terminal, and server play important roles as the main components of this system. Specific embodiments are described below.

[1778] System Configuration

[1779] In this system, users input information using a device (e.g., smartphone, tablet, PC), and the device transmits that information to the server. The server receives the information, retrieves relevant information from the database, analyzes the data, and generates the optimal plan for the user.

[1780] Hardware and software to be used

[1781] User terminal: A device used by a user to input information. Specifically, this includes smartphones, tablets, and personal computers.

[1782] Server: A computer device that receives, preprocesses, analyzes, and sends generated plans based on data. Specific software options include Python, SQL, scikit-learn, pandas, and generative artificial intelligence (e.g., GPT-3).

[1783] Database: Data storage for storing existing user information and related information. SQL databases are often used.

[1784] Data processing and data calculation

[1785] 1. Receiving user input information

[1786] The user inputs their existing information and specific requests through the terminal's interface. For example, they might input a specific request such as, "I want an exercise and meal plan to lose 5kg."

[1787] 2. Transfer and Reception of Information

[1788] The device sends information collected from the user to the server. This transmission is done via an HTTP request.

[1789] The server receives this request and retrieves relevant information (e.g., past health data or purchase history) from the database. Queries to the database are executed using the SQL language.

[1790] 3. Data preprocessing

[1791] The server preprocesses the collected data. Specific processing includes data cleaning (handling missing or outlier values) and data normalization (scaling). These processes are performed using libraries such as Python's pandas library.

[1792] 4. Feature extraction

[1793] The server inputs preprocessed data into a machine learning model and extracts features. The scikit-learn library is used for the specific processing. For example, features such as the user's weight fluctuation patterns and preferred exercise types might be extracted.

[1794] 5. Plan generation using generative artificial intelligence

[1795] The server inputs the extracted features as prompts into a generative artificial intelligence (AI) to generate the optimal plan. The generative AI (e.g., GPT-3) then performs analysis based on these prompts and generates a specific plan.

[1796] 6. Converting and sending the plan format.

[1797] The server converts the generated plan into a specified format (e.g., JSON format) and sends it to the terminal. This process is performed as an HTTP response.

[1798] The device receives this response and notifies the user. Specifically, it notifies the user via smartphone push notifications, etc., that "A new plan has been generated," and displays the plan details within the app.

[1799] Specific example

[1800] For example, by inputting prompt statements like the following into the AI ​​model, it is possible to generate a specific plan based on the user's request:

[1801] Example of a prompt:

[1802] "I want an exercise and meal plan to lose 5kg."

[1803] Based on this prompt, the AI ​​model generates text suggesting a "three-times-a-week-jogging and low-fat meal plan," and the server sends that text to the terminal for the user to receive.

[1804] The above describes an embodiment of the present invention, which allows users to quickly obtain individually customized plans and enables dynamic adjustments based on feedback.

[1805] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1806] Step 1:

[1807] User input phase

[1808] Input: Existing user information or specific requests. Example: "I want an exercise and meal plan to lose 5kg."

[1809] Output: Information entered by the user into the terminal

[1810] Specific operation: The user uses an application on their device (smartphone, computer, etc.) to enter their existing information (e.g., current weight, activity level, etc.) and specific requests into a text field, and then presses the "Send" button. This information is temporarily stored on the device.

[1811] Step 2:

[1812] Data collection phase

[1813] Input: User input information saved on the device

[1814] Output: Information to send to the server

[1815] Specific operation: The terminal packages the user's input information as an HTTP request and sends it to the server. The server receives this request and retrieves relevant information such as past health data and purchase history from the user database using SQL queries.

[1816] Step 3:

[1817] Data preprocessing and feature extraction phase

[1818] Input: User information collected by the server, related database information

[1819] Output: Preprocessed data and features

[1820] Specific operation: The server preprocesses the collected data. First, it cleans the data (e.g., removes outliers, imputes missing values), and then normalizes (scales) it. Next, it extracts features using a machine learning model (e.g., scikit-learn). These features include the user's weight change patterns and preferred exercise types.

[1821] Step 4:

[1822] Plan generation phase

[1823] Input: Preprocessed data and extracted features

[1824] Output: Generated plan

[1825] Specific operation: The server inputs the extracted features as prompts into a generative artificial intelligence model (e.g., GPT-3). The generative AI analyzes these prompts and generates a specific plan. For example, it might generate a plan for "jogging three times a week and a low-fat diet."

[1826] Step 5:

[1827] Plan presentation phase

[1828] Input: Generated plan

[1829] Output: Information presented to the user

[1830] Specific operation: The server converts the generated plan into a predetermined format (such as JSON format) and sends it to the device as an HTTP response. The device receives this response and notifies the user. Specifically, it uses smartphone push notifications to inform the user that "a new plan has been generated" and displays the detailed plan within the app.

[1831] Step 6:

[1832] Dialogue and Feedback Phase

[1833] Input: User feedback

[1834] Output: Updated plan

[1835] Specific operation: The user sends questions and feedback about the generated plan through their device. Specifically, they use an input form within the app to enter feedback such as, "Jogging is difficult. Are there any other exercises?" The device sends this feedback to the server as an HTTP request. The server receives the feedback and, if necessary, inputs new prompts into the generative AI to regenerate the plan. The server then sends the regenerated plan to the device and presents it to the user again. For example, it might suggest, "Cycling three times a week instead."

[1836] Through the above processing steps, users can quickly obtain individually customized plans, and the system allows for dynamic adjustments to those plans based on their feedback.

[1837] (Application Example 1)

[1838] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1839] Modern factories are required to maximize production efficiency while minimizing machine downtime. However, conventional methods lack the means to comprehensively analyze the operating data, inspection information, and current work status of each machine to generate optimal production schedules and work plans. As a result, problems such as inefficient production schedules and unplanned downtime occur. This invention aims to solve these problems and improve the production efficiency of factories.

[1840] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1841] In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and to obtain relevant information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for acquiring machine operation data, inspection information, and current work status in the factory and generating an optimal production schedule and work plan, and means for transmitting the generated production plan as instructions to the factory robots. This makes it possible to improve the overall operational efficiency of the factory and generate an optimal production schedule and work plan that minimizes downtime.

[1842] A "user" is a factory operator who operates the system and inputs the necessary information.

[1843] "Existing information" refers to information that users input into the system, such as product delivery dates, production quantities, and work instructions.

[1844] A "server" is a computer system that receives user information, retrieves related information from a database, and generates the optimal plan.

[1845] A "database" is a system that stores and manages various types of factory data, such as machine operation data, maintenance information, and work history.

[1846] "Preprocessing" refers to the process of cleaning, normalizing, and removing outliers from collected information.

[1847] "Features" are important pieces of information extracted from pre-processed data to optimize production schedules.

[1848] "Generative artificial intelligence" refers to an artificial intelligence system that automatically generates the optimal plan based on the input data.

[1849] An "optimal plan" is a production schedule and work plan designed to maximize factory production efficiency and minimize downtime.

[1850] "Prescribed format" refers to the state in which the generated plan has been converted into a format that can be executed by a terminal or robot.

[1851] A "terminal" is a device used by users to input information or to review generated plans.

[1852] "Feedback" refers to requests for revisions or opinions that users provide regarding the generated plan.

[1853] "Machine operation data in a factory" refers to data on the operating status, usage time, and efficiency of machinery within a factory.

[1854] "Inspection information" refers to data on the maintenance history and maintenance plans of machinery within the factory.

[1855] "Current work status" refers to data on ongoing work and production activities within the factory.

[1856] A "production schedule" is a plan of time and work to efficiently manage and operate a factory's production process.

[1857] A "work plan" is a plan that outlines specific work instructions and procedures.

[1858] A "robot" is an automated machine that performs production activities and tasks within a factory.

[1859] This invention is a system that automatically generates and proposes an optimal production schedule and work plan to the user using generative artificial intelligence (generative AI model) based on factory machine operation data, inspection information, and current work status. The embodiments for carrying out this invention will be described in detail below.

[1860] System Configuration

[1861] This system consists of the following main components:

[1862] 1. Terminal as a user input device

[1863] 2. Servers that collect and process data

[1864] 3. Factory robots that perform specified tasks

[1865] Hardware and software used

[1866] Devices as terminals: Smartphones, tablets, personal computers, etc.

[1867] A powerful computer that acts as a server for data collection, preprocessing, feature extraction, and plan generation.

[1868] Machine learning software: The scikit-learn library in Python is used for data preprocessing, and RandomForestRegressor is used as the generative AI model.

[1869] Robot: An automated machine in a factory that performs a specified production activity.

[1870] Processing flow

[1871] 1. User input phase

[1872] The user inputs requirements such as delivery date and production quantity through the terminal interface. For example, a request might be, "I want to produce 1,000 units of the product by December 31st."

[1873] 2. Data Collection Phase

[1874] The server receives machine operation data, inspection information, and current work status collected within the factory.

[1875] 3. Data preprocessing and feature extraction phase

[1876] The server cleans, normalizes, and removes outliers from the collected data. Specifically, it uses the Python scikit-learn library.

[1877] After the data has been preprocessed, features are extracted. For example, these might include machine downtime or operating efficiency.

[1878] 4. Plan Generation Phase

[1879] The server inputs the extracted features into a generating AI model to automatically generate the optimal production schedule and work plan. Here, RandomForestRegressor is used to generate the plan.

[1880] The generated plan might be something like, "a production schedule that maximizes uptime and includes maintenance three times a week."

[1881] 5. Plan Presentation Phase

[1882] The generated plan is converted to a predetermined format on the server and sent to the terminal. The user reviews the plan through the terminal and sends instructions to the factory robots as needed.

[1883] 6. Dialogue and Feedback Phase

[1884] Users provide feedback on the generated plan. For example, they might say, "I'd like to extend the deadline to January 31st."

[1885] The server regenerates the plan based on this feedback and provides the updated plan to the user.

[1886] Specific example

[1887] 1. If the user is a factory operator

[1888] The operator enters a request stating, "We want to produce 1,000 units of product by December 31st."

[1889] The server collects factory machine operation data, inspection information, and current work status, extracts features, and then generates an optimal production schedule.

[1890] For example, a production schedule that maximizes operating time and includes maintenance three times a week is proposed.

[1891] If an operator provides feedback requesting an extension of the delivery date, the server analyzes the feedback and generates a new schedule.

[1892] Example of a prompt

[1893] For example, an operator might input the following information as an example of health data collection:

[1894] We collect the following information as user health data:

[1895] Weight changes over the past 6 months

[1896] Food diary

[1897] How many times a week do you exercise?

[1898] Please generate the optimal plan to lose 5kg.

[1899] Thus, the present invention can maximize factory production efficiency and eliminate inefficient production schedules and unplanned downtime.

[1900] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1901] Step 1:

[1902] The user inputs existing information through the terminal's interface. This input information includes production requests, such as "We want to produce 1000 units of product by December 31st." The input information is sent directly from the terminal to the server.

[1903] Step 2:

[1904] The server receives existing information sent from the terminal. It also retrieves relevant information from the database. This relevant information includes machine operation data, inspection information, and current work status in the factory. The server collects all of this data together.

[1905] Step 3:

[1906] The server preprocesses the collected information. Preprocessing involves cleaning, normalizing, and removing outliers from the data. Specifically, it uses the Python scikit-learn library to scale and filter the data to create an accurate dataset.

[1907] Step 4:

[1908] The server extracts features from the pre-processed data. These features include, for example, the average machine downtime and operating efficiency. These features are important indicators for optimizing production schedules.

[1909] Step 5:

[1910] The server inputs the extracted features into a generating AI model (RandomForestRegressor) to generate the optimal production schedule and work plan. For example, a production schedule that maximizes uptime and includes three maintenance cycles per week is generated. The generated production plan is temporarily stored on the server.

[1911] Step 6:

[1912] The generated plan is converted to a predetermined format by the server. The converted plan is sent to the terminal and displayed on the user's (factory operator's) screen. The user can review the production plan and modify the work instructions as needed.

[1913] Step 7:

[1914] Users provide feedback through their devices. This feedback may include requests such as, "I would like to extend the delivery date to January 31st." The feedback is sent from the device to the server.

[1915] Step 8:

[1916] The server updates the plan based on the feedback received. The generating AI model analyzes the new feedback information and regenerates the production schedule as needed. For example, a "new production schedule that accommodates the extended delivery date" is generated.

[1917] Step 9:

[1918] The updated plan is resent from the server to the terminal and displayed again to the user. The user reviews the new plan and sends it to the robot as the final work instruction. Once this is complete, the system begins processing the next cycle.

[1919] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1920] This invention relates to a system that combines existing information provided by the user with emotional information read by an emotion engine, automatically generates an optimal plan using generative artificial intelligence, and proposes it to the user. The following describes the specific program processing, its explanation in natural language, and specific examples.

[1921] In this system, the user, terminal, server, and emotion engine work together to generate a plan that is tailored to the user's needs and emotional state. The following processes are performed in an embodiment of the invention.

[1922] Program processing

[1923] 1. User Input Phase

[1924] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[1925] 2. Data Collection Phase

[1926] The terminal receives information entered by the user and sends it to the server. This process takes place in real time.

[1927] 3. Emotional Data Collection Phase

[1928] The device analyzes the user's emotional state using an emotion engine and sends the results to the server. The emotion engine uses facial recognition and voice analysis technologies to determine the emotions the user is currently feeling.

[1929] 4. Data Acquisition and Preprocessing Phase

[1930] The server receives user input information and sentiment data, and retrieves relevant information from the database. For example, it collects past weight fluctuations, food records, and exercise history.

[1931] The server preprocesses the collected information, cleaning the data, removing outliers, and completing incomplete data. At this stage, the data is converted into a consistent format.

[1932] 5. Feature Extraction Phase

[1933] The server extracts features using pre-processed data. Machine learning models are used to identify important attributes and metrics from the data. For example, it might extract user exercise patterns, dietary habits, and allergy information.

[1934] 6. Plan Generation Phase

[1935] The server inputs the extracted features and sentiment data into a generative artificial intelligence system and instructs it to generate a plan that is suitable for the user's needs and current emotional state.

[1936] Generative artificial intelligence analyzes the input data and automatically generates the optimal plan for the user.

[1937] 7. Plan Presentation Phase

[1938] Generative artificial intelligence generates specific exercise and meal plans based on the analysis results. For example, it might suggest a plan for jogging three times a week and a low-fat diet.

[1939] The server converts the generated plan and formats it into the required format before sending it to the user's terminal.

[1940] 8. Dialogue and feedback phase based on user emotions

[1941] The device displays the plan generated for the user. For example, it might allow users to view detailed plans through smartphone notifications.

[1942] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[1943] The device sends this feedback to the server in real time.

[1944] 9. Feedback Analysis and Plan Re-adjustment Phase

[1945] The server receives user feedback and emotional data determined by the emotion engine, and analyzes the information. If necessary, it instructs the generative artificial intelligence to generate a plan under new conditions.

[1946] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might generate a plan suggesting "cycling three times a week instead."

[1947] The server converts the updated plan and sends it back to the terminal.

[1948] 10. Displaying updated plans

[1949] The device will display the plan to the user again and provide updated information.

[1950] Specific example

[1951] The following process is a concrete example of what happens when a user requests a health plan.

[1952] 1. User Input Phase

[1953] The user uses their device to input, "I want an exercise and meal plan to lose 5kg."

[1954] 2. Data Collection Phase

[1955] The terminal sends this input to the server.

[1956] 3. Emotional Data Collection Phase

[1957] The device uses an emotion engine to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[1958] 4. Data Acquisition and Preprocessing Phase

[1959] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database.

[1960] 5. Feature Extraction Phase

[1961] The server removes outliers from the collected data and normalizes the data. Next, it extracts features.

[1962] 6. Plan Generation Phase

[1963] The server inputs feature data and sentiment data into a generative artificial intelligence system to generate a plan that best suits the user's needs and emotions.

[1964] 7. Plan Presentation Phase

[1965] The generative artificial intelligence generates a plan suggesting "three jogging sessions per week and a low-fat diet," and sends it to the terminal via the server.

[1966] 8. Dialogue and feedback phase based on user emotions

[1967] The device displays a plan to the user. The user provides feedback on the plan, such as, "Jogging is difficult. Are there any other exercises?"

[1968] The device sends this feedback to the server.

[1969] 9. Feedback Analysis and Plan Re-adjustment Phase

[1970] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate a plan under new conditions.

[1971] Generative artificial intelligence generates a plan that suggests "cycling three times a week instead."

[1972] The server resends the update plan to the device.

[1973] 10. Displaying updated plans

[1974] The device displays the updated plan to the user.

[1975] This invention allows users to receive more personalized plans according to their emotional state, achieving high user satisfaction by flexibly responding to the user's emotions.

[1976] The following describes the processing flow.

[1977] Step 1:

[1978] The user inputs their existing information and specific requests through the terminal's interface. For example, they might enter a request such as, "I would like an exercise and meal plan to lose 5 kg."

[1979] Step 2:

[1980] The terminal receives user input information and sends it to the server.

[1981] Step 3:

[1982] The device uses a built-in emotion engine to determine the user's current emotional state. This emotion determination is performed in real time based on data such as the user's facial expressions and voice.

[1983] Step 4:

[1984] The device sends emotion data, determined by the emotion engine, to the server. This ensures that the user's input information and emotional state are received by the server.

[1985] Step 5:

[1986] The server analyzes the user's input and retrieves relevant information from the database. For example, it collects data such as past weight changes, dietary records, and exercise history.

[1987] Step 6:

[1988] The server preprocesses the collected information. Preprocessing includes data cleaning (removing outliers, imputing incomplete data) and normalization.

[1989] Step 7:

[1990] The server uses machine learning models to extract key features from pre-processed data. For example, it identifies the user's past exercise patterns, dietary habits, and allergy information.

[1991] Step 8:

[1992] The server inputs the extracted features and sentiment data into a generative artificial intelligence system, instructing it to generate the optimal plan based on the user's needs and current emotional state.

[1993] Step 9:

[1994] Generative artificial intelligence analyzes input data and automatically generates the optimal plan for the user. For example, it might suggest a plan that includes jogging three times a week and a low-fat diet.

[1995] Step 10:

[1996] The server receives the generated plan, converts it to a predetermined format, and then sends it to the terminal.

[1997] Step 11:

[1998] The device displays the generated plan to the user. For example, it provides a detailed plan through smartphone notifications.

[1999] Step 12:

[2000] Users can ask questions or provide feedback about the generated plan. For example, they might enter feedback such as, "Jogging is difficult. Are there any other exercises?"

[2001] Step 13:

[2002] The device receives user feedback and sends it to the server in real time.

[2003] Step 14:

[2004] The server analyzes user feedback and sentiment data, and, if necessary, instructs the generative artificial intelligence to generate plans under new conditions.

[2005] Step 15:

[2006] Generative artificial intelligence considers user feedback and emotional state to generate new plans. For example, it might suggest "cycling three times a week instead."

[2007] Step 16:

[2008] The server converts the updated plan and sends it back to the terminal.

[2009] Step 17:

[2010] The device displays the updated plan to the user and provides the latest plan information.

[2011] Through these steps, the system responds quickly and accurately to user needs and emotions, generating and adjusting plans dynamically. This allows users to receive a personalized plan that is best suited to their situation and feelings.

[2012] (Example 2)

[2013] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2014] Traditional plan generation systems have struggled to provide personalized plans that take into account the user's emotional state. Therefore, there is a need for a system that generates flexible and optimal plans based on the user's current emotions and updates them in real time based on feedback.

[2015] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving existing information and sentiment information transmitted from the user and obtaining relevant information from a database, means for preprocessing the collected information and performing data cleaning and removal of outliers, and means for extracting features using the preprocessed data. This makes it possible to generate an optimal plan tailored to the user's sentiment state and update it in real time as needed.

[2016] A "user" refers to an individual who uses the system to input information and provides feedback on the proposed plan.

[2017] A "terminal" refers to an electronic device used by a user to input information, send the results of an analysis of their emotional state to a server, and receive a generated plan.

[2018] A "server" refers to a computer system that receives data sent by users, retrieves relevant information from a database, and performs data preprocessing, feature extraction, plan generation, and updates.

[2019] An "emotion engine" refers to an automated tool that analyzes a user's facial expressions and voice to obtain emotional information.

[2020] "Existing information" refers to past data and information about the current situation that users input into the system.

[2021] "Related information" refers to information that the server retrieves from the database, including user health data, past exercise history, and dietary records.

[2022] "Preprocessing" refers to the process of performing operations such as cleaning collected data, removing outliers, and supplementing incomplete data.

[2023] "Features" refer to important attributes or metrics extracted from pre-processed data.

[2024] A "generative artificial intelligence model" refers to a machine learning model that automatically generates the optimal plan based on user needs and emotional information.

[2025] "Plan" refers to exercise and dietary suggestions generated by generative artificial intelligence.

[2026] "Feedback" refers to the opinions and requests that users provide regarding the generated plan.

[2027] "Real-time" refers to a state where input and output occur almost simultaneously, with very little latency.

[2028] This invention relates to a system that combines existing information provided by the user with emotional information acquired by an emotion engine, automatically generates an optimal plan using a generative artificial intelligence model, and proposes it to the user. Below, we will describe the specific program processing of the system in natural language, and add specific examples.

[2029] System Configuration

[2030] This system consists of several main components, including the user, terminal, server, emotion engine, and generative artificial intelligence model. The roles and interactions of each component enable the system to automatically generate and present plans based on the user's needs and emotional state.

[2031] Hardware and software configuration

[2032] 1. Terminal:

[2033] These are devices that provide a user interface, such as smartphones, tablets, or PCs.

[2034] The built-in camera and microphone are used to analyze facial expressions and voice using an emotion engine.

[2035] The interface operates within a web browser or a specific application.

[2036] 2. Server:

[2037] It is a high-performance computer system that collects and preprocesses user data and generates plans using a generative artificial intelligence model.

[2038] It has a built-in database that stores and manages relevant information such as the user's health data, past exercise history, and dietary records.

[2039] 3. Emotional Engine:

[2040] This software uses computer vision and speech analysis technologies to analyze the user's emotional state.

[2041] For example, emotions can be identified from the user's facial expressions and voice tone.

[2042] 4. Generative artificial intelligence models:

[2043] For example, advanced generative artificial intelligence such as GPT-3 is used to analyze collected data and generate the optimal plan for the user.

[2044] Specific examples of actions

[2045] 1. User input phase:

[2046] The user enters their desired goals and requirements through the terminal's interface. For example, they might enter, "I want an exercise and meal plan to lose 5kg."

[2047] 2. Data Collection Phase:

[2048] The terminal receives information entered by the user and transmits this information to the server in real time.

[2049] 3. Emotional Data Collection Phase:

[2050] The device uses its built-in emotion engine to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[2051] 4. Data processing and preprocessing:

[2052] The server retrieves the user's existing health data (e.g., past weight changes, dietary records, exercise history) from the database and performs preprocessing. The collected data is cleaned, outliers are removed, and the format is standardized.

[2053] 5. Feature extraction:

[2054] The server extracts features from the pre-processed data and identifies important attributes and metrics. For example, it uses machine learning algorithms to identify exercise patterns, dietary tendencies, allergy information, and so on.

[2055] 6. Plan generation:

[2056] The server inputs extracted features and sentiment data into a generative artificial intelligence model, instructing it to generate an optimal plan based on the user's needs and emotional state. The generative AI model analyzes this data and generates, for example, a "three-times-a-week-jogging and low-fat-meal plan."

[2057] 7. Presentation of the plan and feedback:

[2058] The generative artificial intelligence model sends the generated plan to the terminal via the server, and the terminal displays this plan to the user. The user then provides feedback on this plan.

[2059] The device sends user feedback to the server in real time, and the server analyzes the feedback and sentiment data.

[2060] 8. Readjust the plan:

[2061] The server sends the analysis results to a generative artificial intelligence model and instructs it to generate a plan under new conditions. For example, it might generate a "cycling plan three times a week instead."

[2062] Examples of prompt statements

[2063] For example, the following request is input to a generative artificial intelligence model:

[2064] "I'd like you to suggest an exercise and meal plan to lose 5kg. My current weight is 70kg, and my past exercise history mainly consists of jogging, but I don't have any specific dietary restrictions."

[2065] Through the above process, the present invention makes it possible to provide personalized plans according to the user's emotional state, thereby improving user satisfaction.

[2066] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2067] Step 1:

[2068] User input phase

[2069] The user enters their desired goals and requirements through the terminal's interface.

[2070] Specific input: The user enters, "I would like an exercise and meal plan to lose 5kg."

[2071] The device receives this information and stores it in its internal data structure.

[2072] Input: User's goals and requirements

[2073] Output: Saved user input data

[2074] Step 2:

[2075] Data collection phase

[2076] The terminal sends the information received from the user to the server.

[2077] Specific operation: The terminal sends saved user input data to the server in real time.

[2078] Input: User input data

[2079] Output: User data sent to the server

[2080] Step 3:

[2081] Emotional data collection phase

[2082] The device uses an emotion engine to analyze the user's facial expressions and voice to acquire emotion data.

[2083] Specific operation: Activates the facial recognition camera and microphone, processes the collected data with an emotion engine, and generates emotion data.

[2084] The device sends the acquired emotion data to the server.

[2085] Input: User's facial expression data and voice data

[2086] Output: Sentiment data sent to the server

[2087] Step 4:

[2088] Data acquisition and preprocessing phase

[2089] The server receives existing information and sentiment data sent by the user and retrieves relevant health data from the database.

[2090] Specific actions: Query the database to collect data such as past weight fluctuations, meal records, and exercise history.

[2091] The server preprocesses the collected data, cleaning it, removing outliers, and completing incomplete data.

[2092] Input: Existing user information and sentiment data

[2093] Output: Preprocessed data

[2094] Step 5:

[2095] Feature extraction phase

[2096] The server extracts features using the pre-processed data.

[2097] Specific operation: Machine learning algorithms (e.g., random forest, decision tree, etc.) are used to identify important attributes such as exercise patterns, dietary tendencies, and allergy information.

[2098] Input: Pre-processed data

[2099] Output: Extracted features

[2100] Step 6:

[2101] Plan generation phase

[2102] The server inputs the extracted feature vectors and sentiment data into a generative artificial intelligence model and instructs it to generate the optimal plan.

[2103] Specific operation: A generative artificial intelligence model (e.g., GPT-3) is input with prompt text and feature data to generate a plan.

[2104] Input: Extracted features and sentiment data

[2105] Output: Generated plan

[2106] Step 7:

[2107] Plan presentation phase

[2108] The generative artificial intelligence model sends the generated plan back to the server, which then formats the plan into a predetermined format.

[2109] Specific actions: Convert the generated plan to text format and attach additional information as needed.

[2110] The server sends the plan to the terminal.

[2111] Input: Generated plan

[2112] Output: Plan sent to the terminal

[2113] Step 8:

[2114] Dialogue and feedback phase based on user emotions

[2115] The device displays the plan generated for the user.

[2116] Specific actions: The plan will be displayed on the smartphone or PC screen, and the user will be notified.

[2117] Users provide feedback on the plan and enter it into their device. They might ask questions like, "Jogging is difficult. Are there any other exercises?"

[2118] The device sends this feedback to the server in real time.

[2119] Input: User feedback

[2120] Output: Feedback sent to the server

[2121] Step 9:

[2122] Feedback analysis and plan readjustment phase

[2123] The server analyzes user feedback and sentiment data.

[2124] Specific operation: Analyze feedback data and instruct the generative AI model to generate a plan under new conditions. Example: Generate a new plan that suggests cycling three times a week instead.

[2125] The generative artificial intelligence model generates a plan based on new conditions and sends it to the server.

[2126] Input: Feedback and sentiment data

[2127] Output: Updated plan

[2128] Step 10:

[2129] Displaying updated plans

[2130] The server sends the updated plan to the device.

[2131] The device displays the updated plan to the user.

[2132] Specific action: The updated plan will be displayed again on the smartphone or PC screen.

[2133] Input: Updated plan

[2134] Output: Plan displayed to the user

[2135] (Application Example 2)

[2136] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2137] Previously, the lack of a system to quickly provide personalized plans tailored to users' emotions and needs resulted in low user satisfaction. Furthermore, providing appropriate product recommendations in real time at physical stores was difficult, resulting in an unoptimized user shopping experience. To address these challenges, a system that understands users' emotional states and provides immediate, appropriate recommendations based on that understanding is essential.

[2138] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to input existing information, means for the server to receive existing information transmitted from the user and obtain related information from a database, means for the server to preprocess the collected information and extract features, means for the server to input the preprocessed data into a generative artificial intelligence and generate an optimal plan, means for the server to convert the generated plan into a predetermined format and transmit it to a terminal, means for the user to provide feedback and for the server to update the plan based on the feedback, means for analyzing the user's emotional state using an emotion engine and transmitting the emotional information to the server, and means for displaying optimal product suggestions to the user in the store through a smartphone or smart glasses interface. This enables real-time and personalized product suggestions that correspond to the user's emotional state.

[2139] "Existing information" refers to data and historical information that the user has provided in advance.

[2140] A "terminal" refers to a device that a user uses to input or receive information through an interface. Examples include smartphones and smart glasses.

[2141] A "server" refers to a computer system that processes data sent by users and retrieves and manages related information.

[2142] A "database" refers to a system in which information is stored and managed.

[2143] "Preprocessing" refers to the process of cleansing and normalizing collected data and converting it into a format from which features can be extracted.

[2144] "Features" refer to important attributes and indicators extracted from data, and the underlying data that is analyzed by machine learning models.

[2145] "Generative artificial intelligence" refers to machine learning and artificial intelligence technologies that automatically generate the optimal plan based on collected data and features.

[2146] A "plan" refers to a specific action plan or suggestion proposed by a generative artificial intelligence system based on the user's needs and emotions.

[2147] An "emotion engine" refers to a technology that analyzes a user's facial expressions and voice to determine their emotional state.

[2148] An "interface" refers to the screen or input device that a user uses to interact with a system.

[2149] "Feedback" refers to the reactions and opinions that users provide regarding the system.

[2150] A "smartphone" refers to a mobile communication device that has calling capabilities and a wide range of applications.

[2151] "Smart glasses" refer to a type of wearable device, specifically glasses with a display function, that presents information to the user.

[2152] "Product recommendations" refer to specific product information that is recommended based on the user's needs and emotional state.

[2153] A "physical store" refers to a store that has a physical presence and is a place where users can go in person to purchase products.

[2154] This invention relates to a system that provides personalized in-store product recommendations based on the user's emotional state. The specific configuration and operation of the system are described below.

[2155] Hardware and software to be used

[2156] hardware

[2157] 1. Device: A device used by the user (e.g., smartphone, smart glasses). This includes cameras and microphones.

[2158] 2. Server: A computer system that processes, manages, and runs generative artificial intelligence.

[2159] 3. Database: Stores user information, past purchase history, sentiment data, etc.

[2160] software

[2161] 1. Emotion Engine: A technology that analyzes a user's facial expressions and voice to determine their emotional state (e.g., FaceAPI, Microsoft Emotion API).

[2162] 2. Generative Artificial Intelligence: Machine learning and artificial intelligence technologies that automatically generate optimal plans based on collected data (e.g., GPT-4).

[2163] 3. Database management software: A system for operating and managing databases (e.g., MySQL).

[2164] 4. Machine learning frameworks: Used for data preprocessing, feature extraction, model building, and analysis (e.g., Scikit-learn, TensorFlow).

[2165] System operation

[2166] 1. User input

[2167] Users input their current shopping purpose and product categories of interest via their smartphone or smart glasses.

[2168] Specific example: The user enters "I want new summer clothes" into the device.

[2169] 2. Submitting the entered information

[2170] The terminal sends user input information to the server.

[2171] 3. Analysis of emotional state

[2172] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and sends the emotion data to the server.

[2173] Specific example: Determining a user's level of excitement and interest based on their facial expressions and comments while viewing products in a store.

[2174] 4. Data Collection and Preprocessing

[2175] The server receives user input information and sentiment data, retrieves relevant data (such as past purchase history) from the database, and preprocesses that data. Preprocessing includes data cleansing, normalization, and consistency checks.

[2176] 5. Feature Extraction

[2177] The server extracts key features from pre-processed data using a machine learning model. These features represent the user's purchasing tendencies and current emotional state.

[2178] 6. Generating the optimal plan

[2179] The server inputs extracted feature vectors and sentiment data into a generative artificial intelligence system to generate an optimal product suggestion plan for the user. The generated plan is then converted by the server into a predetermined format and sent to the terminal.

[2180] Specific example: A plan is proposed that says, "New summer casual dresses are now 20% off."

[2181] 7. Plan provision and feedback

[2182] The terminal displays a proposed plan to the user, and the user provides feedback on it. The feedback is sent to the server, which updates the plan based on that feedback.

[2183] Specific example: A user enters feedback saying, "Is this dress available in a different color?"

[2184] Example of a prompt

[2185] "I want some new summer clothes."

[2186] "I'd like to know about any discounts on the products I've shown interest in."

[2187] "I'd like to check if this product is available in other colors."

[2188] These methods enable real-time and personalized product recommendations tailored to the user's emotional state, thereby increasing user satisfaction.

[2189] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2190] Step 1:

[2191] The user uses a device (smartphone or smart glasses) to input their current shopping purpose and product categories of interest. The user's request, "I want new summer clothes," is generated as input data.

[2192] Step 2:

[2193] The terminal sends user input information to the server. The terminal sends a request, which is the input data, and the server receives it. Data transmission occurs in real time.

[2194] Step 3:

[2195] The emotion engine uses the camera and microphone built into the device to analyze the user's facial expressions and voice, and generates emotion data. This emotion data includes emotional information (for example, excitement) determined from the user's facial expressions and statements when viewing a product.

[2196] Step 4:

[2197] The device sends the generated emotion data to the server. The emotion data is sent from the device to the server and used as input information for analysis.

[2198] Step 5:

[2199] The server receives user input information and sentiment data, and retrieves relevant information from the database. This relevant information includes data indicating the user's past purchase history, product preferences, and current sentiment state.

[2200] Step 6:

[2201] The server performs preprocessing on the collected information, removing outliers and normalizing the data. The preprocessed data is then formatted to ensure data consistency and allow for feature extraction.

[2202] Step 7:

[2203] The server inputs pre-processed data into a machine learning model and extracts features. These features include user purchasing tendencies and product categories of interest. Input data (pre-processed data) and output data (features) are generated at this stage.

[2204] Step 8:

[2205] The server inputs feature data and sentiment data into a generative artificial intelligence system to generate an optimal product recommendation plan. The generated plan is based on the user's needs and emotional state. For example, a plan such as "New summer casual dresses, now 20% off" might be generated.

[2206] Step 9:

[2207] The server converts the generated product proposal plan into a predetermined format and sends it to the terminal. The input data (generated plan) is converted into output data (plan in a predetermined format) and sent.

[2208] Step 10:

[2209] The device displays a suggested plan to the user. The user provides feedback on it (e.g., "Is this dress available in a different color?"). This feedback is then generated as new input data.

[2210] Step 11:

[2211] The device sends user feedback to the server in real time. Feedback data is sent and received by the server. This feedback data is then input as new conditions.

[2212] Step 12:

[2213] The server analyzes feedback and emotional data and instructs the generative artificial intelligence to generate plans under new conditions. The generative AI generates new product suggestion plans that match the feedback and emotional state. For example, it might generate a product that suggests cycling three times a week instead.

[2214] Step 13:

[2215] The server converts the updated plan into the specified format and resends it to the terminal. The updated plan is then presented to the user.

[2216] Step 14:

[2217] The device displays updated product plans to the user. This allows the user to see new suggestions and have a more satisfying shopping experience.

[2218] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2219] The data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of the data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2220] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[2221] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2222] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[2223] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[2224] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[2225] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[2226] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[2227] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[2228] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[2229] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[2230] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[2231] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2232] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[2233] The following types of processors can be used as hardware resources to perform specific processing. Ex...

Claims

1. A means for the user to input existing information, A means by which the server receives existing information sent by the user and retrieves related information from the database, The server preprocesses the collected information and extracts features, A means by which a server inputs pre-processed data into a generative artificial intelligence system to generate an optimal plan, A means by which the server converts the generated plan into a predetermined format and sends it to the terminal, A means by which users provide feedback and the server updates the plan based on that feedback, A system that includes this.

2. The system according to claim 1 for processing user health data.

3. The system according to claim 1, wherein the plan generated by the generative artificial intelligence includes an exercise plan and a meal plan based on the user's health data.

Images