system

Abstract

We provide the system. [Solution] Means of entering user information, A means for processing the entered user information and generating a list of necessary procedures, A means of selecting the most suitable vendor based on user requirements, A means of visualizing the procedure list, Means of proposing related products and services, As a means of providing reminders as aftercare, A system that includes this.

Description

Technical Field

[0001] The technology of this disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Moving is a complex process involving many procedures and options. Therefore, it is difficult to know where to request, which procedures should be prioritized, and often forget the procedures required at the new address, making it difficult to proceed efficiently and smoothly. And selecting the goods and services necessary for life in the new residence after moving is also very time-consuming. As a result, there is a problem that many users feel stressed and regard the moving experience as complicated and time-consuming.

Means for Solving the Problems

[0005] This invention provides a system that includes means for inputting user information, processes the input information, and generates a list of necessary procedures. It also includes means for selecting the most suitable contractor based on the user's conditions, and clarifies the priority of procedures by visualizing the procedure list. Furthermore, by including means for suggesting relevant products and services and means for providing reminders as aftercare, the system streamlines the entire moving experience and allows users to start their new life in their new home with peace of mind.

[0006] "User information" refers to information that users enter into the system, including personal information, conditions, and preferences related to moving.

[0007] The "procedure list" is a list containing all the necessary administrative and utility procedures at your new address, designed to ensure that users complete all required steps without missing anything.

[0008] "Service provider selection" is the process of choosing the most suitable moving service provider based on the user's requirements.

[0009] "Visualization" refers to organizing information in a way that is easy for users to understand and representing it visually.

[0010] "Related products and services" refers to furniture, home appliances, internet services, and other items that may be needed for life in a new home.

[0011] A "reminder" refers to a system function that notifies users of deadlines for necessary procedures and important matters so that they do not forget to perform them. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3]It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0013] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0014] First, the language used in the following description will be explained.

[0015] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, the numbered communication I / F (Interface) is an interface that includes a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is implemented as a system to support the moving process. Specifically, it is designed to streamline the procedures required for moving, allowing users to smoothly begin their new life in their new home. The system primarily functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and further suggest products and services necessary for a new life at the new address.

[0034] First, users enter their old and new addresses, planned moving date, and other moving requirements via their smartphone or computer. This information is then transmitted from the device to a server in the cloud via the internet. The information is encrypted and processed securely.

[0035] Next, the server analyzes the received information and lists all procedures related to the user's new address. This is based on a database of local laws and regulations and general moving procedures. This allows for centralized management of procedures that previously required individual research.

[0036] The server then selects moving companies that meet the user's requirements and generates proposals from multiple companies. This allows the user to choose the best company based on cost, ratings, and services offered. The selection results can be viewed on the terminal, and the user can proceed with the contract immediately.

[0037] Furthermore, the server recommends relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. This makes it easy for users to arrange for the items they need to start their new life in their new home.

[0038] Finally, the invention also includes an aftercare function: the server schedules deadlines for important procedures and sends reminder notifications to the user via the terminal. This function prevents users from missing procedures and allows them to start their new life in their new home with peace of mind.

[0039] As a concrete example, consider a user moving to Tokyo. When this user enters their planned moving date and new address, the server lists the necessary items such as "notifying Tokyo residents of their change of address," "contracting for gas and water," and "selecting an internet provider," and presents recommended vendors and service providers. In this way, the present invention is designed to allow users to centrally manage complex and sporadic procedures.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] The user accesses the relocation support system and enters their new address, planned moving date, and any other additional information into the on-screen form. The terminal receives the data entered by the user and prepares to send it to the database.

[0043] Step 2:

[0044] The terminal sends user information to the server. The data being transmitted is encoded to ensure the security of the transmission. The terminal waits for confirmation of receipt from the server.

[0045] Step 3:

[0046] The server analyzes the received user information. It retrieves legal regulations and resident procedures related to the new address from the database and automatically generates the necessary procedures for the user. This list includes procedures for changing addresses and contracting for utilities (electricity, gas, water, etc.).

[0047] Step 4:

[0048] The server searches its database for moving companies based on the user's requirements. It takes into account service details, costs, and past ratings to generate a list of the most suitable companies for the user.

[0049] Step 5:

[0050] The terminal displays a generated list of procedures and information on selecting a moving company to the user. Based on this information, the user can choose a moving company and begin scheduling the necessary tasks.

[0051] Step 6:

[0052] Based on the user's past purchase history and preference data, the server suggests products and services that may be needed at their new address. This includes purchasing furniture and selecting an internet plan.

[0053] Step 7:

[0054] The terminal displays a list of products and services suggested by the server to the user. The user can click on a product or service of interest on the terminal to view details.

[0055] Step 8:

[0056] The server sets reminders for important procedures. The terminal sends a reminder notification to the user at the specified date and time to help the user remember the procedure.

[0057] (Example 1)

[0058] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0059] The procedures and related tasks involved in moving are complex, making it difficult for users to research and manage them individually. Furthermore, the wide variety of moving companies and necessary goods and services at the new residence makes it challenging for users to make optimal choices. In addition, managing deadlines for necessary procedures is not easy. A system is needed to address these problems and enable users to move efficiently and smoothly.

[0060] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0061] In this invention, the server includes means for generating a list of necessary procedures using a generated AI model based on user information, means including an algorithm for selecting the most suitable service provider based on user conditions, and means for managing deadlines for important procedures and providing reminder notifications to the user. This enables the user to proceed with moving procedures efficiently. Furthermore, by selecting the most suitable service provider and recommending related products and services, the user can move smoothly.

[0062] "User information" refers to information necessary when moving, such as the user's old and new addresses, planned moving date, and any special requirements.

[0063] A "generative AI model" is an artificial intelligence system that uses pattern recognition and machine learning based on user information to generate an optimal list of procedures.

[0064] A "service provider" is a company or organization that provides services related to moving, and is selected based on the user's requirements.

[0065] An "algorithm" is a computational method that systematizes the procedures for selecting the most suitable vendor based on user conditions and past evaluations.

[0066] "Preference information" refers to data that indicates an individual user's preferences and needs, based on their past selection and purchase history.

[0067] A "reminder notification" is a notification sent from a server to inform the user in advance of procedures or necessary actions they should take.

[0068] An "interface" refers to the user interface or connection method that allows a user to interact with a system and exchange information through their device.

[0069] This invention is a system that allows users to efficiently manage the moving process and smoothly start their new life. The system primarily has functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and suggest necessary goods and services at the new address.

[0070] Users enter their old and new addresses and moving date from their smartphones or computers. This entered information is transmitted via the internet to a server in the cloud. The data is encrypted using the SSL / TLS protocol to ensure security.

[0071] The server uses a generative AI model to analyze information received from the user and automatically generate a list of necessary procedures. This model refers to a regional legal and regulatory database to cover all procedures required at the user's new address. It also selects multiple moving companies based on the user's specified conditions, and the AI ​​algorithm generates these suggestions. Furthermore, the server analyzes the user's preference data and past purchase history to recommend products and services that will enrich their life at their new address.

[0072] The system manages the user's procedure schedule and sends reminders via the terminal when deadlines for important procedures are approaching. This feature helps users prevent missing procedures.

[0073] As a concrete example, a user might enter a prompt such as, "I'm planning to move in March, and my new address is in Tokyo. Please tell me what procedures I need to follow." Based on this information, the server creates a list of necessary procedures, such as "Notifying Tokyo of your change of address," "Contracting for gas and water services," and "Selecting an internet provider," and recommends the most suitable companies and products. This allows the user to easily manage the complex procedures related to moving and efficiently begin their new life.

[0074] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0075] Step 1:

[0076] Users enter their old and new addresses, planned moving date, and any special requirements using a smartphone or computer. The entered information is sent from the device to a server in the cloud. The data is encrypted using the SSL / TLS protocol during transmission, ensuring its security. The input consists of the user profile and moving conditions, which form the basis for generating procedures in the next step.

[0077] Step 2:

[0078] The server uses a generating AI model to analyze the received user information and generate a list of necessary procedures. The AI ​​model references regional legal and regulatory databases to cover all legal and administrative procedures related to the new address. It performs data calculations to list each procedure based on the local rule map and the new address. The output is a generated list of procedures, comprehensively covering all relevant items.

[0079] Step 3:

[0080] The server executes an algorithm to select the most suitable moving company based on the user's specified conditions. This algorithm generates a list of candidate companies based on evaluation criteria including ratings, cost, and service scope. The algorithm matches company information with the user's conditions, and the AI ​​guides the optimal selection. The output of this process is a list of candidate companies.

[0081] Step 4:

[0082] The server uses a generative AI model to analyze user preference data and past purchase history to recommend products and services that will enrich life at the new address. The analysis uses a product database and the user's past purchase information to list the most suitable products based on the user's interests. The output is a list of recommended products and services.

[0083] Step 5:

[0084] The server manages critical deadlines at each step of the procedure and sends reminder notifications to the user via their device at the specified date and time. This ensures the user completes the procedure without missing deadlines. Notifications are sent via email or in-app notifications and include procedure details and deadline information. The output is the scheduled notification.

[0085] (Application Example 1)

[0086] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] Moving requires numerous procedures and gathering information about the new living environment. However, these tasks are often very complicated, and choosing restaurants around the new address presents challenges, such as information overload and the difficulty of making appropriate choices within the limited time immediately after moving.

[0088] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0089] In this invention, the server includes means for inputting user information, means for processing the input user information and generating a list of necessary procedures, means for selecting the most suitable service provider based on the user's conditions, means for visualizing the list of procedures, means for suggesting related products and services, means for acquiring restaurant information based on the user's location information at their new address and generating meal suggestions based on the user's preferences, and means for providing reminders as aftercare. This not only efficiently manages procedures related to moving but also enables appropriate meal choices around the new address.

[0090] "User information" refers to personal data that users provide in order to handle procedures related to moving and to begin life at their new address.

[0091] A "procedure list" is a list of various procedures required for a user to move, and is generated based on local laws and regulations and general procedures.

[0092] "Vendor selection" is the process of choosing the most suitable moving company based on the user's requirements.

[0093] "Visualization" refers to presenting the generated list of procedures in an easy-to-understand manner to the user, enabling them to visually understand the progress of the procedures and the necessary actions.

[0094] "Suggestions for related products and services" refers to recommending suitable products and services based on the user's preferences and past purchase history in order to enrich their life at their new address.

[0095] "Location information" refers to geographical information about the area where a user is newly moving, and is used to improve the accuracy of suggestions.

[0096] "Restaurant information" refers to data about restaurants located near the user's new address, and is acquired and used as part of the recommendation process.

[0097] "Meal suggestions" refer to presenting appropriate meal options at the user's new address based on their preferences, and are suggestions aimed at improving the comfort of their life.

[0098] A "reminder" is a notification method that alerts users to deadlines for important procedures and events related to moving.

[0099] The system that realizes this invention is server-centric and operates in conjunction with the user's terminal. The user uses a terminal such as a smartphone or computer to input information necessary for moving, such as old and new addresses and moving date. This information is encrypted and transmitted to the server via the internet. Upon receiving this information, the server generates a list of necessary procedures based on local legal regulations and a database of general moving procedures.

[0100] The server also selects the most suitable moving company based on the user's requirements. It generates proposals from multiple companies and makes them available for review on the user's device. This allows the user to choose the best company while comparing costs, services, and other factors. Next, the server collects restaurant information based on the user's location to generate meal suggestions that reflect past preferences, in order to help the user choose meals related to their new address. This uses external data sources such as restaurant information APIs.

[0101] Furthermore, the server suggests relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. It also includes an after-care function that sends reminder notifications for important deadlines and events. Notifications are sent via SMS and email using the Twilio API, among others.

[0102] As a concrete example, let's assume a user is moving to Tokyo. The user enters their new address, and the server visualizes procedures such as "notifying the authorities of their change of address" and "contracting for water and gas." Furthermore, it suggests delivery information for popular nearby restaurants and restaurants related to the user's favorites.

[0103] An example of a prompt for the generating AI model is, "Please suggest some recommended restaurants and eateries for when I move to my new address {address}." This mechanism aims to comprehensively improve the user experience when moving.

[0104] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0105] Step 1:

[0106] The user enters their old and new addresses and moving date using their device. The entered information is encrypted and sent to the server via the internet. The system retrieves address information as input data and processes it to transfer securely.

[0107] Step 2:

[0108] The server uses the received user information to refer to a database of regional laws, regulations, and general procedures, and generates a list of necessary procedures. At this stage, data retrieval and listing are performed using the entered address information to obtain the procedure list output.

[0109] Step 3:

[0110] The server selects multiple moving companies based on the user's criteria and generates proposals from each company. Based on the conditions pre-set by the user, the server executes a company selection algorithm and generates a list of companies. This information is formatted for viewing on the terminal.

[0111] Step 4:

[0112] The server collects local restaurant data via an API based on the user's new address and generates meal suggestions by combining it with the user's preference data. Specifically, it performs data mining using location data as input to obtain output in the form of restaurant information and recommended menu items.

[0113] Step 5:

[0114] The server analyzes user preference data and past purchase history to suggest products and services relevant to life at the new address. It uses preference data as input to perform data calculations and apply product recommendation algorithms to output a list of related products.

[0115] Step 6:

[0116] The server generates reminders for important procedures and events, and notifies users via SMS or email using the Twilio API. The reminder generation process utilizes a scheduling function that takes schedule information and due date data as input and generates a notification message as output.

[0117] An example of a prompt would be, "Please suggest some restaurants or eateries in my new address {address}."

[0118] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0119] This invention combines emotion recognition technology with a system that supports the user's moving process to provide optimized services tailored to the user's emotions. In addition to functions such as user information input, information processing, procedure list generation, contractor selection, service proposal, and aftercare, this system is equipped with an emotion engine that grasps the user's emotions in real time and adjusts service proposals based on emotional aspects.

[0120] First, the user inputs basic information via a device such as a smartphone or computer and provides emotional data through emotion sensors (such as a camera or voice input). The device then acquires this data and prepares to send it to the cloud.

[0121] The server analyzes emotional data sent by the user and uses an emotion engine to recognize the user's current emotional state. This allows for, for example, the suggestion of special services to reduce stress or advice on procedures that are sensitive to the user's emotions. For instance, it can suggest procedures to help a user who is feeling stressed to help them relax.

[0122] Furthermore, the server uses user sentiment data to tailor the selection of moving companies based on those sentiments. In particular, it offers users who are experiencing stress the option to prioritize the selection of fast and reputable companies.

[0123] The products and services suggested are also tailored to the user's emotional state through the emotional engine. For example, a user who is emotionally exhausted will be recommended furniture and services that prioritize comfort.

[0124] Furthermore, the server adjusts reminders to the optimal timing and content based on the user's emotions, and delivers them to the user through their device. This system allows users to perform important procedures at a time when they are most emotionally receptive.

[0125] As a concrete example, when a user moves during a busy period, the system uses an emotion engine to detect the user's stress level and recommends moving companies and methods to expedite the process. At the same time, to ensure the user can complete the move without undue stress, the reminders sent are concise, helpful, and contain minimal notifications to reduce the burden. In this way, the system aims to provide emotional support and reduce stress throughout the user's moving process.

[0126] The following describes the processing flow.

[0127] Step 1:

[0128] Users log in to the relocation support system using their devices and enter basic profile information and detailed information about their move. They also provide emotional data through sensors such as cameras and microphones.

[0129] Step 2:

[0130] The terminal collects user information entered and emotional data acquired from sensors in real time, and transmits this data to a server. The data is encoded to ensure security and is efficiently transferred.

[0131] Step 3:

[0132] The server analyzes the received user data, retrieves the procedures the user needs at their new address from the database, and generates a list of necessary procedures. At this time, emotional data is also analyzed to understand the user's current emotional state.

[0133] Step 4:

[0134] The server considers emotional data to create optimal procedures tailored to the user's emotional state and offers suggestions to help reduce stress. For example, if the user's emotional state is unstable, it will prioritize recommending simple and easy procedures.

[0135] Step 5:

[0136] The server lists the most suitable moving companies based on the user's conditions and emotional state. For example, it recommends highly-rated companies and those capable of quick response times to users who are experiencing stress.

[0137] Step 6:

[0138] The terminal displays a generated list of procedures and proposals from moving companies to the user. The user can review the proposals and proceed with the procedures with the company they are interested in.

[0139] Step 7:

[0140] The server uses an emotion engine to suggest the most relevant products and services to the user. These suggestions are adjusted based on the user's emotional state.

[0141] Step 8:

[0142] The device will notify the user of the aforementioned proposals, allowing the user to review the options before proceeding with a purchase or contract.

[0143] Step 9:

[0144] The server takes the user's emotional state into consideration and generates reminders appropriate to the timing of the procedure. The reminders are adjusted to the user's most receptive timing, notifying them of important procedures and deadlines.

[0145] Step 10:

[0146] The device notifies the user of the generated reminder. The user receives the notification and can complete the task according to their schedule.

[0147] (Example 2)

[0148] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0149] During the moving process, the inability to provide optimal service that reflects the user's emotional state can cause stress to the user. Furthermore, the progress of the procedures may not be aligned with the user's situation, leading to problems such as unnecessary time and effort being required. This results in challenges that prevent the moving process from proceeding efficiently and comfortably.

[0150] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0151] In this invention, the server includes means for collecting user information and analyzing emotional data, means for adjusting a list of procedures based on the emotional analysis results, and means for selecting and providing the most suitable service provider or service according to the user's emotions. This makes it possible to provide services that are appropriate to the user's emotional state, optimize the entire moving process, and realize a less burdensome experience for the user.

[0152] "User information" refers to basic personal data and conditions related to moving, including address, date, and desired conditions for the new residence.

[0153] "Emotional data" refers to information that quantifies or shows the user's psychological state, including data such as facial expressions and voice tone obtained using a camera or microphone.

[0154] "Emotional analysis" refers to the process of evaluating a user's current psychological state based on collected emotional data and identifying emotional states such as stress and reassurance.

[0155] A "procedure list" is a compilation of various specific steps and necessary items related to moving, serving as a guide for users to efficiently prepare for their move.

[0156] "Related services" refer to additional tasks and support provided to ensure a smooth and comfortable moving process, and include services such as cleaning, packing, and transportation.

[0157] A "reminder" is a notification that informs the user of necessary actions at a desired time, a function designed to prevent them from missing important procedures.

[0158] A "generative AI model" refers to an artificial intelligence technology used for natural language processing and data analysis, specifically a model designed to support user sentiment analysis and the optimization of service recommendations.

[0159] A "prompt sentence" is a natural language sentence input to give instructions to a generative AI model, serving as a trigger to obtain specific information or perform specific processing.

[0160] This invention is a system that supports the moving process while taking user emotions into consideration, and begins with inputting user information and emotional data using a device such as a smartphone or computer.

[0161] Users enter basic information such as their address and moving date via their device. Simultaneously, sensors such as the device's built-in camera and microphone provide emotional data, including facial expressions and voice. For example, if a user is experiencing stress, this emotional data is collected through their facial expressions and tone of voice.

[0162] The device sends collected information and emotional data to the cloud. On this cloud, data analysis is performed using a generative AI model. An emotional analysis engine operates, analyzing the user's psychological state in real time. The generative AI model is also used in the process of optimizing the next steps and services for the user.

[0163] Based on the results of the user's emotion analysis, the server can suggest services that are sensitive to the user's feelings and select the most suitable moving company. For example, if the user is feeling anxious, it will prioritize suggesting appropriate support to provide reassurance. Furthermore, it will also suggest products and services that reflect these results, recommending furniture and services that take the user's emotional state into consideration.

[0164] Furthermore, the server adjusts reminders based on the user's mental state, notifying them of important procedures at the appropriate time. This feature is designed to make the relocation process a more comfortable experience for the user.

[0165] As a concrete example, the entire system will operate by inputting the following prompt message into the generating AI model.

[0166] "My emotional sensors are detecting a high level of stress regarding my move. Please suggest procedures that will allow me to proceed efficiently while remaining relaxed."

[0167] Thus, this system aims to provide a user-friendly, stress-free moving process by offering optimal service suggestions and adjustments based on emotional data.

[0168] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0169] Step 1:

[0170] The user enters basic information about their move through the device. This includes their address, desired moving date, and requirements for their new home. Simultaneously, the device's camera and microphone are used to collect emotional data such as the user's facial expressions and voice. The device preprocesses this entered information and emotional data, converting it into a format suitable for transmission.

[0171] Step 2:

[0172] The terminal sends pre-processed information to a server in the cloud. Here, the entered user information and sentiment data are transferred to the server. The terminal's role is to transmit this information securely while maintaining data integrity.

[0173] Step 3:

[0174] The server analyzes the received emotional data using an AI model generated on the cloud. The emotional analysis engine then runs to identify the user's emotional state. The emotional data received as input is classified into states such as "reassured," "anxious," and "stressed" through computational processing using artificial intelligence technology. These analysis results are then used to propose the most suitable services.

[0175] Step 4:

[0176] The server generates a list of procedures and related services that are appropriate for the user's psychological state based on the analysis results. The generating AI model optimizes these and outputs content that is tailored to the user. Specifically, it provides procedures to enhance a sense of security and suggests products that have a relaxing effect.

[0177] Step 5:

[0178] The server selects the most suitable service provider for the user based on the analyzed emotional data. For users experiencing stress, it prioritizes and recommends service providers that are fast and have a good reputation. This selection process takes the user's psychological state into consideration.

[0179] Step 6:

[0180] The server creates reminders based on the user's status and prepares to send notifications at the appropriate time. The content and timing of the reminders are adjusted based on the prompt messages. For example, a suggestion such as, "We recommend you get some rest tonight in preparation for tomorrow's cargo removal," might be made.

[0181] Step 7:

[0182] The device notifies the user of reminders sent from the server. The notifications the user receives are emotionally sensitive and include advice tailored to the user's progress in moving preparations. This helps the user to proceed with the moving process smoothly.

[0183] (Application Example 2)

[0184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0185] Providing appropriate customer service based on customer emotions is difficult, and there is a need for means to realize services that align with the diverse emotional states of customers. Conventional technologies have been unable to accurately grasp customer emotions and provide services accordingly, limiting the improvement of customer satisfaction.

[0186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0187] In this invention, the server includes a device for acquiring user information, a device for processing the acquired user information and generating a list of necessary processing steps, and a device for selecting the most suitable service provider based on the user's status. This enables real-time analysis of customer sentiment data and the provision of optimal services based on the results.

[0188] A "device for acquiring user information" is a device for acquiring basic information and emotional data from users, thereby enabling the collection of detailed data about the user.

[0189] A "device that processes acquired user information and generates a list of necessary processes" is a device that analyzes and processes information acquired from users and lists appropriate procedures and service processes tailored to each user.

[0190] A "device that selects the optimal provider based on the user's condition" is a device that takes into account the user's emotions and condition and has the function of selecting the most appropriate service provider for the user.

[0191] A "processing list visualization device" is a device that displays a list of generated procedures and service processes to the user in an easy-to-understand manner, playing a role in helping the user understand them.

[0192] A "device that suggests related products and services" is a device that recommends products and services suitable for a user based on the user's data and emotional state.

[0193] A "device that provides notifications as aftercare" is a device that provides users with important information and reminders at an appropriate time after they have used a service.

[0194] A "device for acquiring user emotional data" is a device that collects a user's emotional state using devices such as cameras and microphones.

[0195] A "device for analyzing acquired emotional data" is a device that processes acquired emotional data and recognizes the user's current emotional state.

[0196] A "device that optimizes customer service methods and product suggestions based on emotional data" is a device that selects and suggests the most suitable customer service methods and products for a user based on analyzed emotional data.

[0197] This invention realizes a system in which a user, a terminal, and a server work together to provide optimal services based on emotions. When a user visits a store, smart glasses, which act as a terminal, are worn, and emotional data is acquired from the user's facial expressions and voice tone using a camera and microphone. The acquired data is transmitted to the server via an internet connection. The server analyzes the data using emotion recognition software, such as Microsoft® Azure® facial recognition API, to understand the user's emotional state. Based on the analysis results, the server executes a program that selects the most suitable customer service method, related products, and services for the customer, and transmits the results to the terminal in real time. The terminal visualizes the customer service method and product suggestions based on the analysis results on its display. For example, a store employee wearing smart glasses may receive a notification such as "We recommend the new product promotion" when they recognize a customer's smile. An example of a prompt sentence using a generative AI model is, "Please suggest the best product for a customer who wants to relax." In this way, the system of the present invention can understand the user's emotions in real time and improve customer satisfaction through optimal service suggestions.

[0198] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0199] Step 1:

[0200] A user enters a store, and a store employee wearing smart glasses (which act as a terminal) begins assisting the customer. The smart glasses acquire the user's facial expressions and voice data in real time using a camera and microphone. This acquired data serves as input and is processed in the next step.

[0201] Step 2:

[0202] The device transmits the acquired facial expression and voice data to a server in the cloud via the internet. The server uses Microsoft Azure's facial recognition API to analyze the emotional data. Here, data calculations are performed to recognize the user's emotional state by analyzing the facial expression data and voice tone. The user's emotional state data is output as a result of the analysis.

[0203] Step 3:

[0204] Based on the analysis results, the server generates prompt messages using a generative AI model. At this time, prompt messages such as "Please suggest the best products for a customer who wants to relax" are generated. Based on these prompt messages, the server lists the most suitable products and services for the customer.

[0205] Step 4:

[0206] The product and service suggestions selected by the server are transmitted to the terminal in real time. The terminal receives this information and displays it on the smart glasses' display. This display is the output, and specific customer service methods and product suggestions are provided to the store staff as instructions.

[0207] Step 5:

[0208] Based on the suggestions accepted by the user, the store clerk explains product details to the customer and provides related services. Specific actions by the clerk include bringing the product to the customer's table and demonstrating it. Further feedback is obtained on the spot through interaction with the customer, and additional information is resent from the terminal to the server as needed.

[0209] In this way, users, devices, and servers work together to provide optimal service suggestions tailored to customer emotions, thereby improving the user experience.

[0210] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0211] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0212] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0213] [Second Embodiment]

[0214] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0215] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0216] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0217] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0218] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0220] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0221] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0222] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0223] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0224] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0225] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0226] This invention is implemented as a system to support the moving process. Specifically, it is designed to streamline the procedures required for moving, allowing users to smoothly begin their new life in their new home. The system primarily functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and further suggest products and services necessary for a new life at the new address.

[0227] First, users enter their old and new addresses, planned moving date, and other moving requirements via their smartphone or computer. This information is then transmitted from the device to a server in the cloud via the internet. The information is encrypted and processed securely.

[0228] Next, the server analyzes the received information and lists all procedures related to the user's new address. This is based on a database of local laws and regulations and general moving procedures. This allows for centralized management of procedures that previously required individual research.

[0229] The server then selects moving companies that meet the user's requirements and generates proposals from multiple companies. This allows the user to choose the best company based on cost, ratings, and services offered. The selection results can be viewed on the terminal, and the user can proceed with the contract immediately.

[0230] Furthermore, the server recommends relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. This makes it easy for users to arrange for the items they need to start their new life in their new home.

[0231] Finally, the invention also includes an aftercare function: the server schedules deadlines for important procedures and sends reminder notifications to the user via the terminal. This function prevents users from missing procedures and allows them to start their new life in their new home with peace of mind.

[0232] As a concrete example, consider a user moving to Tokyo. When this user enters their planned moving date and new address, the server lists the necessary items such as "notifying Tokyo residents of their change of address," "contracting for gas and water," and "selecting an internet provider," and presents recommended vendors and service providers. In this way, the present invention is designed to allow users to centrally manage complex and sporadic procedures.

[0233] The following describes the processing flow.

[0234] Step 1:

[0235] The user accesses the relocation support system and enters their new address, planned moving date, and any other additional information into the on-screen form. The terminal receives the data entered by the user and prepares to send it to the database.

[0236] Step 2:

[0237] The terminal sends user information to the server. The data being transmitted is encoded to ensure the security of the transmission. The terminal waits for confirmation of receipt from the server.

[0238] Step 3:

[0239] The server analyzes the received user information. It retrieves legal regulations and resident procedures related to the new address from the database and automatically generates the necessary procedures for the user. This list includes procedures for changing addresses and contracting for utilities (electricity, gas, water, etc.).

[0240] Step 4:

[0241] The server searches its database for moving companies based on the user's requirements. It takes into account service details, costs, and past ratings to generate a list of the most suitable companies for the user.

[0242] Step 5:

[0243] The terminal displays a generated list of procedures and information on selecting a moving company to the user. Based on this information, the user can choose a moving company and begin scheduling the necessary tasks.

[0244] Step 6:

[0245] Based on the user's past purchase history and preference data, the server suggests products and services that may be needed at their new address. This includes purchasing furniture and selecting an internet plan.

[0246] Step 7:

[0247] The terminal displays a list of products and services suggested by the server to the user. The user can click on a product or service of interest on the terminal to view details.

[0248] Step 8:

[0249] The server sets reminders for important procedures. The terminal sends a reminder notification to the user at the specified date and time to help the user remember the procedure.

[0250] (Example 1)

[0251] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0252] The procedures and related tasks involved in moving are complex, making it difficult for users to research and manage them individually. Furthermore, the wide variety of moving companies and necessary goods and services at the new residence makes it challenging for users to make optimal choices. In addition, managing deadlines for necessary procedures is not easy. A system is needed to address these problems and enable users to move efficiently and smoothly.

[0253] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0254] In this invention, the server includes means for generating a list of necessary procedures using a generated AI model based on user information, means including an algorithm for selecting the most suitable service provider based on user conditions, and means for managing deadlines for important procedures and providing reminder notifications to the user. This enables the user to proceed with moving procedures efficiently. Furthermore, by selecting the most suitable service provider and recommending related products and services, the user can move smoothly.

[0255] "User information" refers to information necessary when moving, such as the user's old and new addresses, planned moving date, and any special requirements.

[0256] A "generative AI model" is an artificial intelligence system that uses pattern recognition and machine learning based on user information to generate an optimal list of procedures.

[0257] A "service provider" is a company or organization that provides services related to moving, and is selected based on the user's requirements.

[0258] An "algorithm" is a computational method that systematizes the procedures for selecting the most suitable vendor based on user conditions and past evaluations.

[0259] "Preference information" refers to data that indicates an individual user's preferences and needs, based on their past selection and purchase history.

[0260] A "reminder notification" is a notification sent from a server to inform the user in advance of procedures or necessary actions they should take.

[0261] An "interface" refers to the user interface or connection method that allows a user to interact with a system and exchange information through their device.

[0262] This invention is a system that allows users to efficiently manage the moving process and smoothly start their new life. The system primarily has functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and suggest necessary goods and services at the new address.

[0263] Users enter their old and new addresses and moving date from their smartphones or computers. This entered information is transmitted via the internet to a server in the cloud. The data is encrypted using the SSL / TLS protocol to ensure security.

[0264] The server uses a generative AI model to analyze information received from the user and automatically generate a list of necessary procedures. This model refers to a regional legal and regulatory database to cover all procedures required at the user's new address. It also selects multiple moving companies based on the user's specified conditions, and the AI ​​algorithm generates these suggestions. Furthermore, the server analyzes the user's preference data and past purchase history to recommend products and services that will enrich their life at their new address.

[0265] The system manages the user's procedure schedule and sends reminders via the terminal when deadlines for important procedures are approaching. This feature helps users prevent missing procedures.

[0266] As a concrete example, a user might enter a prompt such as, "I'm planning to move in March, and my new address is in Tokyo. Please tell me what procedures I need to follow." Based on this information, the server creates a list of necessary procedures, such as "Notifying Tokyo of your change of address," "Contracting for gas and water services," and "Selecting an internet provider," and recommends the most suitable companies and products. This allows the user to easily manage the complex procedures related to moving and efficiently begin their new life.

[0267] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0268] Step 1:

[0269] Users enter their old and new addresses, planned moving date, and any special requirements using a smartphone or computer. The entered information is sent from the device to a server in the cloud. The data is encrypted using the SSL / TLS protocol during transmission, ensuring its security. The input consists of the user profile and moving conditions, which form the basis for generating procedures in the next step.

[0270] Step 2:

[0271] The server uses a generating AI model to analyze the received user information and generate a list of necessary procedures. The AI ​​model references regional legal and regulatory databases to cover all legal and administrative procedures related to the new address. It performs data calculations to list each procedure based on the local rule map and the new address. The output is a generated list of procedures, comprehensively covering all relevant items.

[0272] Step 3:

[0273] The server executes an algorithm to select the most suitable moving company based on the user's specified conditions. This algorithm generates a list of candidate companies based on evaluation criteria including ratings, cost, and service scope. The algorithm matches company information with the user's conditions, and the AI ​​guides the optimal selection. The output of this process is a list of candidate companies.

[0274] Step 4:

[0275] The server uses a generative AI model to analyze user preference data and past purchase history to recommend products and services that will enrich life at the new address. The analysis uses a product database and the user's past purchase information to list the most suitable products based on the user's interests. The output is a list of recommended products and services.

[0276] Step 5:

[0277] The server manages critical deadlines at each step of the procedure and sends reminder notifications to the user via their device at the specified date and time. This ensures the user completes the procedure without missing deadlines. Notifications are sent via email or in-app notifications and include procedure details and deadline information. The output is the scheduled notification.

[0278] (Application Example 1)

[0279] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0280] Moving requires numerous procedures and gathering information about the new living environment. However, these tasks are often very complicated, and choosing restaurants around the new address presents challenges, such as information overload and the difficulty of making appropriate choices within the limited time immediately after moving.

[0281] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0282] In this invention, the server includes means for inputting user information, means for processing the input user information and generating a list of necessary procedures, means for selecting the most suitable service provider based on the user's conditions, means for visualizing the list of procedures, means for suggesting related products and services, means for acquiring restaurant information based on the user's location information at their new address and generating meal suggestions based on the user's preferences, and means for providing reminders as aftercare. This not only efficiently manages procedures related to moving but also enables appropriate meal choices around the new address.

[0283] "User information" refers to personal data that users provide in order to handle procedures related to moving and to begin life at their new address.

[0284] The "procedure list" is a list that itemizes various procedures necessary for the user's move, and is a list generated based on local regulations and general procedures for each region.

[0285] "Vendor selection" is a process of selecting the most suitable moving vendor based on the user's conditions.

[0286] "Visualization" refers to presenting the generated procedure list to the user in an easy-to-understand manner, and is for enabling the user to visually understand the progress of the procedures and the necessary actions.

[0287] "Recommendation of related products and services" refers to recommending suitable products and services based on the user's preferences and past purchase history in order to enrich the user's life at the new residence.

[0288] "Location information" is geographical information about the area where the user newly moves, and is data used to improve the accuracy of the recommendation.

[0289] "Restaurant information" is data about stores that provide meals around the user's new residence, and is information acquired and used as part of the recommendation.

[0290] "Meal recommendation" is to present appropriate meal options at the new residence of the moving destination based on the user's preferences, and is a recommendation for improving the comfort of life.

[0291] "Reminder" is a notification means to alert the user of the deadlines of important procedures and events related to the move.

[0292] The system that realizes this invention is server-centric and operates in conjunction with the user's terminal. The user uses a terminal such as a smartphone or computer to input information necessary for moving, such as old and new addresses and moving date. This information is encrypted and transmitted to the server via the internet. Upon receiving this information, the server generates a list of necessary procedures based on local legal regulations and a database of general moving procedures.

[0293] The server also selects the most suitable moving company based on the user's requirements. It generates proposals from multiple companies and makes them available for review on the user's device. This allows the user to choose the best company while comparing costs, services, and other factors. Next, the server collects restaurant information based on the user's location to generate meal suggestions that reflect past preferences, in order to help the user choose meals related to their new address. This uses external data sources such as restaurant information APIs.

[0294] Furthermore, the server suggests relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. It also includes an after-care function that sends reminder notifications for important deadlines and events. Notifications are sent via SMS and email using the Twilio API, among others.

[0295] As a concrete example, let's assume a user is moving to Tokyo. The user enters their new address, and the server visualizes procedures such as "notifying the authorities of their change of address" and "contracting for water and gas." Furthermore, it suggests delivery information for popular nearby restaurants and restaurants related to the user's favorites.

[0296] An example of a prompt for the generating AI model is, "Please suggest some recommended restaurants and eateries for when I move to my new address {address}." This mechanism aims to comprehensively improve the user experience when moving.

[0297] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0298] Step 1:

[0299] The user uses the terminal to input the old and new addresses and the planned moving date. The input information is encrypted and transmitted to the server via the Internet. Perform the process of obtaining address information as input data and transferring it securely.

[0300] Step 2:

[0301] Based on the received user information, the server refers to the database on local regulations and general procedures for each region and generates a list of necessary procedures. At this stage, perform data search and listing to obtain an output of a procedure list using the input address information.

[0302] Step 3:

[0303] The server selects multiple moving companies based on the user's conditions and generates proposal information from each company. Execute the algorithm for selecting companies based on the conditions pre-set by the user to generate a list of companies. This information is formatted in a form that can be confirmed on the terminal.

[0304] Step 4:

[0305] The server collects local restaurant data based on the user's new address through the API and generates meal proposals in combination with the user's preference data. Specifically, perform data mining to obtain outputs of restaurant information and recommended menus using location information data as input.

[0306] Step 5:

[0307] The server analyzes the user's preference data and past purchase history and proposes products and services related to life in the new address. Perform data calculation and apply the product recommendation algorithm to output a list of related products using preference data as input.

[0308] Step 6:

[0309] The server generates reminders for important procedures and events, and notifies users via SMS or email using the Twilio API. The reminder generation process utilizes a scheduling function that takes schedule information and due date data as input and generates a notification message as output.

[0310] An example of a prompt would be, "Please suggest some restaurants or eateries in my new address {address}."

[0311] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0312] This invention combines emotion recognition technology with a system that supports the user's moving process to provide optimized services tailored to the user's emotions. In addition to functions such as user information input, information processing, procedure list generation, contractor selection, service proposal, and aftercare, this system is equipped with an emotion engine that grasps the user's emotions in real time and adjusts service proposals based on emotional aspects.

[0313] First, the user inputs basic information via a device such as a smartphone or computer and provides emotional data through emotion sensors (such as a camera or voice input). The device then acquires this data and prepares to send it to the cloud.

[0314] The server analyzes emotional data sent by the user and uses an emotion engine to recognize the user's current emotional state. This allows for, for example, the suggestion of special services to reduce stress or advice on procedures that are sensitive to the user's emotions. For instance, it can suggest procedures to help a user who is feeling stressed to help them relax.

[0315] Furthermore, the server uses user sentiment data to tailor the selection of moving companies based on those sentiments. In particular, it offers users who are experiencing stress the option to prioritize the selection of fast and reputable companies.

[0316] The products and services suggested are also tailored to the user's emotional state through the emotional engine. For example, a user who is emotionally exhausted will be recommended furniture and services that prioritize comfort.

[0317] Furthermore, the server adjusts reminders to the optimal timing and content based on the user's emotions, and delivers them to the user through their device. This system allows users to perform important procedures at a time when they are most emotionally receptive.

[0318] As a concrete example, when a user moves during a busy period, the system uses an emotion engine to detect the user's stress level and recommends moving companies and methods to expedite the process. At the same time, to ensure the user can complete the move without undue stress, the reminders sent are concise, helpful, and contain minimal notifications to reduce the burden. In this way, the system aims to provide emotional support and reduce stress throughout the user's moving process.

[0319] The following describes the processing flow.

[0320] Step 1:

[0321] Users log in to the relocation support system using their devices and enter basic profile information and detailed information about their move. They also provide emotional data through sensors such as cameras and microphones.

[0322] Step 2:

[0323] The terminal collects user information entered and emotional data acquired from sensors in real time, and transmits this data to a server. The data is encoded to ensure security and is efficiently transferred.

[0324] Step 3:

[0325] The server analyzes the received user data, retrieves the procedures the user needs at their new address from the database, and generates a list of necessary procedures. At this time, emotional data is also analyzed to understand the user's current emotional state.

[0326] Step 4:

[0327] The server considers emotional data to create optimal procedures tailored to the user's emotional state and offers suggestions to help reduce stress. For example, if the user's emotional state is unstable, it will prioritize recommending simple and easy procedures.

[0328] Step 5:

[0329] The server lists the most suitable moving companies based on the user's conditions and emotional state. For example, it recommends highly-rated companies and those capable of quick response times to users who are experiencing stress.

[0330] Step 6:

[0331] The terminal displays a generated list of procedures and proposals from moving companies to the user. The user can review the proposals and proceed with the procedures with the company they are interested in.

[0332] Step 7:

[0333] The server uses an emotion engine to suggest the most relevant products and services to the user. These suggestions are adjusted based on the user's emotional state.

[0334] Step 8:

[0335] The device will notify the user of the aforementioned proposals, allowing the user to review the options before proceeding with a purchase or contract.

[0336] Step 9:

[0337] The server takes the user's emotional state into consideration and generates reminders appropriate to the timing of the procedure. The reminders are adjusted to the user's most receptive timing, notifying them of important procedures and deadlines.

[0338] Step 10:

[0339] The device notifies the user of the generated reminder. The user receives the notification and can complete the task according to their schedule.

[0340] (Example 2)

[0341] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0342] During the moving process, the inability to provide optimal service that reflects the user's emotional state can cause stress to the user. Furthermore, the progress of the procedures may not be aligned with the user's situation, leading to problems such as unnecessary time and effort being required. This results in challenges that prevent the moving process from proceeding efficiently and comfortably.

[0343] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0344] In this invention, the server includes means for collecting user information and analyzing emotional data, means for adjusting a list of procedures based on the emotional analysis results, and means for selecting and providing the most suitable service provider or service according to the user's emotions. This makes it possible to provide services that are appropriate to the user's emotional state, optimize the entire moving process, and realize a less burdensome experience for the user.

[0345] "User information" refers to basic personal data and conditions related to moving, including address, date, and desired conditions for the new residence.

[0346] "Emotional data" refers to information that quantifies or shows the user's psychological state, including data such as facial expressions and voice tone obtained using a camera or microphone.

[0347] "Emotional analysis" refers to the process of evaluating a user's current psychological state based on collected emotional data and identifying emotional states such as stress and reassurance.

[0348] A "procedure list" is a compilation of various specific steps and necessary items related to moving, serving as a guide for users to efficiently prepare for their move.

[0349] "Related services" refer to additional tasks and support provided to ensure a smooth and comfortable moving process, and include services such as cleaning, packing, and transportation.

[0350] A "reminder" is a notification that informs the user of necessary actions at a desired time, a function designed to prevent them from missing important procedures.

[0351] A "generative AI model" refers to an artificial intelligence technology used for natural language processing and data analysis, specifically a model designed to support user sentiment analysis and the optimization of service recommendations.

[0352] A "prompt sentence" is a natural language sentence input to give instructions to a generative AI model, serving as a trigger to obtain specific information or perform specific processing.

[0353] This invention is a system that supports the moving process while taking user emotions into consideration, and begins with inputting user information and emotional data using a device such as a smartphone or computer.

[0354] Users enter basic information such as their address and moving date via their device. Simultaneously, sensors such as the device's built-in camera and microphone provide emotional data, including facial expressions and voice. For example, if a user is experiencing stress, this emotional data is collected through their facial expressions and tone of voice.

[0355] The device sends collected information and emotional data to the cloud. On this cloud, data analysis is performed using a generative AI model. An emotional analysis engine operates, analyzing the user's psychological state in real time. The generative AI model is also used in the process of optimizing the next steps and services for the user.

[0356] Based on the results of the user's emotion analysis, the server can suggest services that are sensitive to the user's feelings and select the most suitable moving company. For example, if the user is feeling anxious, it will prioritize suggesting appropriate support to provide reassurance. Furthermore, it will also suggest products and services that reflect these results, recommending furniture and services that take the user's emotional state into consideration.

[0357] Furthermore, the server adjusts reminders based on the user's mental state, notifying them of important procedures at the appropriate time. This feature is designed to make the relocation process a more comfortable experience for the user.

[0358] As a concrete example, the entire system will operate by inputting the following prompt message into the generating AI model.

[0359] "My emotional sensors are detecting a high level of stress regarding my move. Please suggest procedures that will allow me to proceed efficiently while remaining relaxed."

[0360] Thus, this system aims to provide a user-friendly, stress-free moving process by offering optimal service suggestions and adjustments based on emotional data.

[0361] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0362] Step 1:

[0363] The user enters basic information about their move through the device. This includes their address, desired moving date, and requirements for their new home. Simultaneously, the device's camera and microphone are used to collect emotional data such as the user's facial expressions and voice. The device preprocesses this entered information and emotional data, converting it into a format suitable for transmission.

[0364] Step 2:

[0365] The terminal sends pre-processed information to a server in the cloud. Here, the entered user information and sentiment data are transferred to the server. The terminal's role is to transmit this information securely while maintaining data integrity.

[0366] Step 3:

[0367] The server analyzes the received emotional data using an AI model generated on the cloud. The emotional analysis engine then runs to identify the user's emotional state. The emotional data received as input is classified into states such as "reassured," "anxious," and "stressed" through computational processing using artificial intelligence technology. These analysis results are then used to propose the most suitable services.

[0368] Step 4:

[0369] The server generates a list of procedures and related services that are appropriate for the user's psychological state based on the analysis results. The generating AI model optimizes these and outputs content that is tailored to the user. Specifically, it provides procedures to enhance a sense of security and suggests products that have a relaxing effect.

[0370] Step 5:

[0371] The server selects the most suitable service provider for the user based on the analyzed emotional data. For users experiencing stress, it prioritizes and recommends service providers that are fast and have a good reputation. This selection process takes the user's psychological state into consideration.

[0372] Step 6:

[0373] The server creates reminders based on the user's status and prepares to send notifications at the appropriate time. The content and timing of the reminders are adjusted based on the prompt messages. For example, a suggestion such as, "We recommend you get some rest tonight in preparation for tomorrow's cargo removal," might be made.

[0374] Step 7:

[0375] The device notifies the user of reminders sent from the server. The notifications the user receives are emotionally sensitive and include advice tailored to the user's progress in moving preparations. This helps the user to proceed with the moving process smoothly.

[0376] (Application Example 2)

[0377] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0378] Providing appropriate customer service based on customer emotions is difficult, and there is a need for means to realize services that align with the diverse emotional states of customers. Conventional technologies have been unable to accurately grasp customer emotions and provide services accordingly, limiting the improvement of customer satisfaction.

[0379] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0380] In this invention, the server includes a device for acquiring user information, a device for processing the acquired user information and generating a list of necessary processing steps, and a device for selecting the most suitable service provider based on the user's status. This enables real-time analysis of customer sentiment data and the provision of optimal services based on the results.

[0381] A "device for acquiring user information" is a device for acquiring basic information and emotional data from users, thereby enabling the collection of detailed data about the user.

[0382] A "device that processes acquired user information and generates a list of necessary processes" is a device that analyzes and processes information acquired from users and lists appropriate procedures and service processes tailored to each user.

[0383] A "device that selects the optimal provider based on the user's condition" is a device that takes into account the user's emotions and condition and has the function of selecting the most appropriate service provider for the user.

[0384] A "processing list visualization device" is a device that displays a list of generated procedures and service processes to the user in an easy-to-understand manner, playing a role in helping the user understand them.

[0385] A "device that suggests related products and services" is a device that recommends products and services suitable for a user based on the user's data and emotional state.

[0386] A "device that provides notifications as aftercare" is a device that provides users with important information and reminders at an appropriate time after they have used a service.

[0387] A "device for acquiring user emotional data" is a device that collects a user's emotional state using devices such as cameras and microphones.

[0388] A "device for analyzing acquired emotional data" is a device that processes acquired emotional data and recognizes the user's current emotional state.

[0389] A "device that optimizes customer service methods and product suggestions based on emotional data" is a device that selects and suggests the most suitable customer service methods and products for a user based on analyzed emotional data.

[0390] This invention realizes a system in which a user, a terminal, and a server work together to provide optimal services based on emotions. When a user visits a store, smart glasses, which act as a terminal, are worn, and emotional data is acquired from the user's facial expressions and voice tone using a camera and microphone. The acquired data is transmitted to the server via an internet connection. The server analyzes the data using emotion recognition software, such as Microsoft Azure's facial recognition API, to understand the user's emotional state. Based on the analysis results, the server executes a program that selects the most suitable customer service method, related products, and services for the customer, and transmits the results to the terminal in real time. The terminal visualizes the customer service method and product suggestions based on the analysis results on its display. For example, a store employee wearing smart glasses may receive a notification such as "We recommend the new product promotion" when they recognize a customer's smile. An example of a prompt sentence using a generative AI model is, "Please suggest the best product for a customer who wants to relax." In this way, the system of the present invention can understand the user's emotions in real time and improve customer satisfaction through optimal service suggestions.

[0391] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0392] Step 1:

[0393] A user enters a store, and a store employee wearing smart glasses (which act as a terminal) begins assisting the customer. The smart glasses acquire the user's facial expressions and voice data in real time using a camera and microphone. This acquired data serves as input and is processed in the next step.

[0394] Step 2:

[0395] The device transmits the acquired facial expression and voice data to a server in the cloud via the internet. The server uses Microsoft Azure's facial recognition API to analyze the emotional data. Here, data calculations are performed to recognize the user's emotional state by analyzing the facial expression data and voice tone. The user's emotional state data is output as a result of the analysis.

[0396] Step 3:

[0397] Based on the analysis results, the server generates prompt messages using a generative AI model. At this time, prompt messages such as "Please suggest the best products for a customer who wants to relax" are generated. Based on these prompt messages, the server lists the most suitable products and services for the customer.

[0398] Step 4:

[0399] The product and service suggestions selected by the server are transmitted to the terminal in real time. The terminal receives this information and displays it on the smart glasses' display. This display is the output, and specific customer service methods and product suggestions are provided to the store staff as instructions.

[0400] Step 5:

[0401] Based on the suggestions accepted by the user, the store clerk explains product details to the customer and provides related services. Specific actions by the clerk include bringing the product to the customer's table and demonstrating it. Further feedback is obtained on the spot through interaction with the customer, and additional information is resent from the terminal to the server as needed.

[0402] In this way, users, devices, and servers work together to provide optimal service suggestions tailored to customer emotions, thereby improving the user experience.

[0403] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0404] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0405] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0406] [Third Embodiment]

[0407] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0408] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0409] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0410] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0411] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0412] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0413] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0414] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0415] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0416] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0417] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0418] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0419] This invention is implemented as a system to support the moving process. Specifically, it is designed to streamline the procedures required for moving, allowing users to smoothly begin their new life in their new home. The system primarily functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and further suggest products and services necessary for a new life at the new address.

[0420] First, users enter their old and new addresses, planned moving date, and other moving requirements via their smartphone or computer. This information is then transmitted from the device to a server in the cloud via the internet. The information is encrypted and processed securely.

[0421] Next, the server analyzes the received information and lists all procedures related to the user's new address. This is based on a database of local laws and regulations and general moving procedures. This allows for centralized management of procedures that previously required individual research.

[0422] The server then selects moving companies that meet the user's requirements and generates proposals from multiple companies. This allows the user to choose the best company based on cost, ratings, and services offered. The selection results can be viewed on the terminal, and the user can proceed with the contract immediately.

[0423] Furthermore, the server recommends relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. This makes it easy for users to arrange for the items they need to start their new life in their new home.

[0424] Finally, the invention also includes an aftercare function: the server schedules deadlines for important procedures and sends reminder notifications to the user via the terminal. This function prevents users from missing procedures and allows them to start their new life in their new home with peace of mind.

[0425] As a concrete example, consider a user moving to Tokyo. When this user enters their planned moving date and new address, the server lists the necessary items such as "notifying Tokyo residents of their change of address," "contracting for gas and water," and "selecting an internet provider," and presents recommended vendors and service providers. In this way, the present invention is designed to allow users to centrally manage complex and sporadic procedures.

[0426] The following describes the processing flow.

[0427] Step 1:

[0428] The user accesses the relocation support system and enters their new address, planned moving date, and any other additional information into the on-screen form. The terminal receives the data entered by the user and prepares to send it to the database.

[0429] Step 2:

[0430] The terminal sends user information to the server. The data being transmitted is encoded to ensure the security of the transmission. The terminal waits for confirmation of receipt from the server.

[0431] Step 3:

[0432] The server analyzes the received user information. It retrieves legal regulations and resident procedures related to the new address from the database and automatically generates the necessary procedures for the user. This list includes procedures for changing addresses and contracting for utilities (electricity, gas, water, etc.).

[0433] Step 4:

[0434] The server searches its database for moving companies based on the user's requirements. It takes into account service details, costs, and past ratings to generate a list of the most suitable companies for the user.

[0435] Step 5:

[0436] The terminal displays a generated list of procedures and information on selecting a moving company to the user. Based on this information, the user can choose a moving company and begin scheduling the necessary tasks.

[0437] Step 6:

[0438] Based on the user's past purchase history and preference data, the server suggests products and services that may be needed at their new address. This includes purchasing furniture and selecting an internet plan.

[0439] Step 7:

[0440] The terminal displays a list of products and services suggested by the server to the user. The user can click on a product or service of interest on the terminal to view details.

[0441] Step 8:

[0442] The server sets reminders for important procedures. The terminal sends a reminder notification to the user at the specified date and time to help the user remember the procedure.

[0443] (Example 1)

[0444] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0445] The procedures and related tasks involved in moving are complex, making it difficult for users to research and manage them individually. Furthermore, the wide variety of moving companies and necessary goods and services at the new residence makes it challenging for users to make optimal choices. In addition, managing deadlines for necessary procedures is not easy. A system is needed to address these problems and enable users to move efficiently and smoothly.

[0446] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0447] In this invention, the server includes means for generating a list of necessary procedures using a generated AI model based on user information, means including an algorithm for selecting the most suitable service provider based on user conditions, and means for managing deadlines for important procedures and providing reminder notifications to the user. This enables the user to proceed with moving procedures efficiently. Furthermore, by selecting the most suitable service provider and recommending related products and services, the user can move smoothly.

[0448] "User information" refers to information necessary when moving, such as the user's old and new addresses, planned moving date, and any special requirements.

[0449] A "generative AI model" is an artificial intelligence system that uses pattern recognition and machine learning based on user information to generate an optimal list of procedures.

[0450] A "service provider" is a company or organization that provides services related to moving, and is selected based on the user's requirements.

[0451] An "algorithm" is a computational method that systematizes the procedures for selecting the most suitable vendor based on user conditions and past evaluations.

[0452] "Preference information" refers to data that indicates an individual user's preferences and needs, based on their past selection and purchase history.

[0453] A "reminder notification" is a notification sent from a server to inform the user in advance of procedures or necessary actions they should take.

[0454] An "interface" refers to the user interface or connection method that allows a user to interact with a system and exchange information through their device.

[0455] This invention is a system that allows users to efficiently manage the moving process and smoothly start their new life. The system primarily has functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and suggest necessary goods and services at the new address.

[0456] Users enter their old and new addresses and moving date from their smartphones or computers. This entered information is transmitted via the internet to a server in the cloud. The data is encrypted using the SSL / TLS protocol to ensure security.

[0457] The server uses a generative AI model to analyze information received from the user and automatically generate a list of necessary procedures. This model refers to a regional legal and regulatory database to cover all procedures required at the user's new address. It also selects multiple moving companies based on the user's specified conditions, and the AI ​​algorithm generates these suggestions. Furthermore, the server analyzes the user's preference data and past purchase history to recommend products and services that will enrich their life at their new address.

[0458] The system manages the user's procedure schedule and sends reminders via the terminal when deadlines for important procedures are approaching. This feature helps users prevent missing procedures.

[0459] As a concrete example, a user might enter a prompt such as, "I'm planning to move in March, and my new address is in Tokyo. Please tell me what procedures I need to follow." Based on this information, the server creates a list of necessary procedures, such as "Notifying Tokyo of your change of address," "Contracting for gas and water services," and "Selecting an internet provider," and recommends the most suitable companies and products. This allows the user to easily manage the complex procedures related to moving and efficiently begin their new life.

[0460] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0461] Step 1:

[0462] Users enter their old and new addresses, planned moving date, and any special requirements using a smartphone or computer. The entered information is sent from the device to a server in the cloud. The data is encrypted using the SSL / TLS protocol during transmission, ensuring its security. The input consists of the user profile and moving conditions, which form the basis for generating procedures in the next step.

[0463] Step 2:

[0464] The server uses a generating AI model to analyze the received user information and generate a list of necessary procedures. The AI ​​model references regional legal and regulatory databases to cover all legal and administrative procedures related to the new address. It performs data calculations to list each procedure based on the local rule map and the new address. The output is a generated list of procedures, comprehensively covering all relevant items.

[0465] Step 3:

[0466] The server executes an algorithm to select the most suitable moving company based on the user's specified conditions. This algorithm generates a list of candidate companies based on evaluation criteria including ratings, cost, and service scope. The algorithm matches company information with the user's conditions, and the AI ​​guides the optimal selection. The output of this process is a list of candidate companies.

[0467] Step 4:

[0468] The server uses a generative AI model to analyze user preference data and past purchase history to recommend products and services that will enrich life at the new address. The analysis uses a product database and the user's past purchase information to list the most suitable products based on the user's interests. The output is a list of recommended products and services.

[0469] Step 5:

[0470] The server manages critical deadlines at each step of the procedure and sends reminder notifications to the user via their device at the specified date and time. This ensures the user completes the procedure without missing deadlines. Notifications are sent via email or in-app notifications and include procedure details and deadline information. The output is the scheduled notification.

[0471] (Application Example 1)

[0472] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0473] Moving requires numerous procedures and gathering information about the new living environment. However, these tasks are often very complicated, and choosing restaurants around the new address presents challenges, such as information overload and the difficulty of making appropriate choices within the limited time immediately after moving.

[0474] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0475] In this invention, the server includes means for inputting user information, means for processing the input user information and generating a list of necessary procedures, means for selecting the most suitable service provider based on the user's conditions, means for visualizing the list of procedures, means for suggesting related products and services, means for acquiring restaurant information based on the user's location information at their new address and generating meal suggestions based on the user's preferences, and means for providing reminders as aftercare. This not only efficiently manages procedures related to moving but also enables appropriate meal choices around the new address.

[0476] "User information" refers to personal data that users provide in order to handle procedures related to moving and to begin life at their new address.

[0477] A "procedure list" is a list of various procedures required for a user to move, and is generated based on local laws and regulations and general procedures.

[0478] "Vendor selection" is the process of choosing the most suitable moving company based on the user's requirements.

[0479] "Visualization" refers to presenting the generated list of procedures in an easy-to-understand manner to the user, enabling them to visually understand the progress of the procedures and the necessary actions.

[0480] "Suggestions for related products and services" refers to recommending suitable products and services based on the user's preferences and past purchase history in order to enrich their life at their new address.

[0481] "Location information" refers to geographical information about the area where a user is newly moving, and is used to improve the accuracy of suggestions.

[0482] "Restaurant information" refers to data about restaurants located near the user's new address, and is acquired and used as part of the recommendation process.

[0483] "Meal suggestions" refer to presenting appropriate meal options at the user's new address based on their preferences, and are suggestions aimed at improving the comfort of their life.

[0484] A "reminder" is a notification method that alerts users to deadlines for important procedures and events related to moving.

[0485] The system that realizes this invention is server-centric and operates in conjunction with the user's terminal. The user uses a terminal such as a smartphone or computer to input information necessary for moving, such as old and new addresses and moving date. This information is encrypted and transmitted to the server via the internet. Upon receiving this information, the server generates a list of necessary procedures based on local legal regulations and a database of general moving procedures.

[0486] The server also selects the most suitable moving company based on the user's requirements. It generates proposals from multiple companies and makes them available for review on the user's device. This allows the user to choose the best company while comparing costs, services, and other factors. Next, the server collects restaurant information based on the user's location to generate meal suggestions that reflect past preferences, in order to help the user choose meals related to their new address. This uses external data sources such as restaurant information APIs.

[0487] Furthermore, the server suggests relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. It also includes an after-care function that sends reminder notifications for important deadlines and events. Notifications are sent via SMS and email using the Twilio API, among others.

[0488] As a concrete example, let's assume a user is moving to Tokyo. The user enters their new address, and the server visualizes procedures such as "notifying the authorities of their change of address" and "contracting for water and gas." Furthermore, it suggests delivery information for popular nearby restaurants and restaurants related to the user's favorites.

[0489] An example of a prompt for the generating AI model is, "Please suggest some recommended restaurants and eateries for when I move to my new address {address}." This mechanism aims to comprehensively improve the user experience when moving.

[0490] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0491] Step 1:

[0492] The user enters their old and new addresses and moving date using their device. The entered information is encrypted and sent to the server via the internet. The system retrieves address information as input data and processes it to transfer securely.

[0493] Step 2:

[0494] The server uses the received user information to refer to a database of regional laws, regulations, and general procedures, and generates a list of necessary procedures. At this stage, data retrieval and listing are performed using the entered address information to obtain the procedure list output.

[0495] Step 3:

[0496] The server selects multiple moving companies based on the user's criteria and generates proposals from each company. Based on the conditions pre-set by the user, the server executes a company selection algorithm and generates a list of companies. This information is formatted for viewing on the terminal.

[0497] Step 4:

[0498] The server collects local restaurant data via an API based on the user's new address and generates meal suggestions by combining it with the user's preference data. Specifically, it performs data mining using location data as input to obtain output in the form of restaurant information and recommended menu items.

[0499] Step 5:

[0500] The server analyzes user preference data and past purchase history to suggest products and services relevant to life at the new address. It uses preference data as input to perform data calculations and apply product recommendation algorithms to output a list of related products.

[0501] Step 6:

[0502] The server generates reminders for important procedures and events, and notifies users via SMS or email using the Twilio API. The reminder generation process utilizes a scheduling function that takes schedule information and due date data as input and generates a notification message as output.

[0503] An example of a prompt would be, "Please suggest some restaurants or eateries in my new address {address}."

[0504] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0505] This invention combines emotion recognition technology with a system that supports the user's moving process to provide optimized services tailored to the user's emotions. In addition to functions such as user information input, information processing, procedure list generation, contractor selection, service proposal, and aftercare, this system is equipped with an emotion engine that grasps the user's emotions in real time and adjusts service proposals based on emotional aspects.

[0506] First, the user inputs basic information via a device such as a smartphone or computer and provides emotional data through emotion sensors (such as a camera or voice input). The device then acquires this data and prepares to send it to the cloud.

[0507] The server analyzes emotional data sent by the user and uses an emotion engine to recognize the user's current emotional state. This allows for, for example, the suggestion of special services to reduce stress or advice on procedures that are sensitive to the user's emotions. For instance, it can suggest procedures to help a user who is feeling stressed to help them relax.

[0508] Furthermore, the server uses user sentiment data to tailor the selection of moving companies based on those sentiments. In particular, it offers users who are experiencing stress the option to prioritize the selection of fast and reputable companies.

[0509] The products and services suggested are also tailored to the user's emotional state through the emotional engine. For example, a user who is emotionally exhausted will be recommended furniture and services that prioritize comfort.

[0510] Furthermore, the server adjusts reminders to the optimal timing and content based on the user's emotions, and delivers them to the user through their device. This system allows users to perform important procedures at a time when they are most emotionally receptive.

[0511] As a concrete example, when a user moves during a busy period, the system uses an emotion engine to detect the user's stress level and recommends moving companies and methods to expedite the process. At the same time, to ensure the user can complete the move without undue stress, the reminders sent are concise, helpful, and contain minimal notifications to reduce the burden. In this way, the system aims to provide emotional support and reduce stress throughout the user's moving process.

[0512] The following describes the processing flow.

[0513] Step 1:

[0514] Users log in to the relocation support system using their devices and enter basic profile information and detailed information about their move. They also provide emotional data through sensors such as cameras and microphones.

[0515] Step 2:

[0516] The terminal collects user information entered and emotional data acquired from sensors in real time, and transmits this data to a server. The data is encoded to ensure security and is efficiently transferred.

[0517] Step 3:

[0518] The server analyzes the received user data, retrieves the procedures the user needs at their new address from the database, and generates a list of necessary procedures. At this time, emotional data is also analyzed to understand the user's current emotional state.

[0519] Step 4:

[0520] The server considers emotional data to create optimal procedures tailored to the user's emotional state and offers suggestions to help reduce stress. For example, if the user's emotional state is unstable, it will prioritize recommending simple and easy procedures.

[0521] Step 5:

[0522] The server lists the most suitable moving companies based on the user's conditions and emotional state. For example, it recommends highly-rated companies and those capable of quick response times to users who are experiencing stress.

[0523] Step 6:

[0524] The terminal displays a generated list of procedures and proposals from moving companies to the user. The user can review the proposals and proceed with the procedures with the company they are interested in.

[0525] Step 7:

[0526] The server uses an emotion engine to suggest the most relevant products and services to the user. These suggestions are adjusted based on the user's emotional state.

[0527] Step 8:

[0528] The device will notify the user of the aforementioned proposals, allowing the user to review the options before proceeding with a purchase or contract.

[0529] Step 9:

[0530] The server takes the user's emotional state into consideration and generates reminders appropriate to the timing of the procedure. The reminders are adjusted to the user's most receptive timing, notifying them of important procedures and deadlines.

[0531] Step 10:

[0532] The device notifies the user of the generated reminder. The user receives the notification and can complete the task according to their schedule.

[0533] (Example 2)

[0534] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0535] During the moving process, the inability to provide optimal service that reflects the user's emotional state can cause stress to the user. Furthermore, the progress of the procedures may not be aligned with the user's situation, leading to problems such as unnecessary time and effort being required. This results in challenges that prevent the moving process from proceeding efficiently and comfortably.

[0536] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0537] In this invention, the server includes means for collecting user information and analyzing emotional data, means for adjusting a list of procedures based on the emotional analysis results, and means for selecting and providing the most suitable service provider or service according to the user's emotions. This makes it possible to provide services that are appropriate to the user's emotional state, optimize the entire moving process, and realize a less burdensome experience for the user.

[0538] "User information" refers to basic personal data and conditions related to moving, including address, date, and desired conditions for the new residence.

[0539] "Emotional data" refers to information that quantifies or shows the user's psychological state, including data such as facial expressions and voice tone obtained using a camera or microphone.

[0540] "Emotional analysis" refers to the process of evaluating a user's current psychological state based on collected emotional data and identifying emotional states such as stress and reassurance.

[0541] A "procedure list" is a compilation of various specific steps and necessary items related to moving, serving as a guide for users to efficiently prepare for their move.

[0542] "Related services" refer to additional tasks and support provided to ensure a smooth and comfortable moving process, and include services such as cleaning, packing, and transportation.

[0543] A "reminder" is a notification that informs the user of necessary actions at a desired time, a function designed to prevent them from missing important procedures.

[0544] A "generative AI model" refers to an artificial intelligence technology used for natural language processing and data analysis, specifically a model designed to support user sentiment analysis and the optimization of service recommendations.

[0545] A "prompt sentence" is a natural language sentence input to give instructions to a generative AI model, serving as a trigger to obtain specific information or perform specific processing.

[0546] This invention is a system that supports the moving process while taking user emotions into consideration, and begins with inputting user information and emotional data using a device such as a smartphone or computer.

[0547] Users enter basic information such as their address and moving date via their device. Simultaneously, sensors such as the device's built-in camera and microphone provide emotional data, including facial expressions and voice. For example, if a user is experiencing stress, this emotional data is collected through their facial expressions and tone of voice.

[0548] The device sends collected information and emotional data to the cloud. On this cloud, data analysis is performed using a generative AI model. An emotional analysis engine operates, analyzing the user's psychological state in real time. The generative AI model is also used in the process of optimizing the next steps and services for the user.

[0549] Based on the results of the user's emotion analysis, the server can suggest services that are sensitive to the user's feelings and select the most suitable moving company. For example, if the user is feeling anxious, it will prioritize suggesting appropriate support to provide reassurance. Furthermore, it will also suggest products and services that reflect these results, recommending furniture and services that take the user's emotional state into consideration.

[0550] Furthermore, the server adjusts reminders based on the user's mental state, notifying them of important procedures at the appropriate time. This feature is designed to make the relocation process a more comfortable experience for the user.

[0551] As a concrete example, the entire system will operate by inputting the following prompt message into the generating AI model.

[0552] "My emotional sensors are detecting a high level of stress regarding my move. Please suggest procedures that will allow me to proceed efficiently while remaining relaxed."

[0553] Thus, this system aims to provide a user-friendly, stress-free moving process by offering optimal service suggestions and adjustments based on emotional data.

[0554] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0555] Step 1:

[0556] The user enters basic information about their move through the device. This includes their address, desired moving date, and requirements for their new home. Simultaneously, the device's camera and microphone are used to collect emotional data such as the user's facial expressions and voice. The device preprocesses this entered information and emotional data, converting it into a format suitable for transmission.

[0557] Step 2:

[0558] The terminal sends pre-processed information to a server in the cloud. Here, the entered user information and sentiment data are transferred to the server. The terminal's role is to transmit this information securely while maintaining data integrity.

[0559] Step 3:

[0560] The server analyzes the received emotional data using an AI model generated on the cloud. The emotional analysis engine then runs to identify the user's emotional state. The emotional data received as input is classified into states such as "reassured," "anxious," and "stressed" through computational processing using artificial intelligence technology. These analysis results are then used to propose the most suitable services.

[0561] Step 4:

[0562] The server generates a list of procedures and related services that are appropriate for the user's psychological state based on the analysis results. The generating AI model optimizes these and outputs content that is tailored to the user. Specifically, it provides procedures to enhance a sense of security and suggests products that have a relaxing effect.

[0563] Step 5:

[0564] The server selects the most suitable service provider for the user based on the analyzed emotional data. For users experiencing stress, it prioritizes and recommends service providers that are fast and have a good reputation. This selection process takes the user's psychological state into consideration.

[0565] Step 6:

[0566] The server creates reminders based on the user's status and prepares to send notifications at the appropriate time. The content and timing of the reminders are adjusted based on the prompt messages. For example, a suggestion such as, "We recommend you get some rest tonight in preparation for tomorrow's cargo removal," might be made.

[0567] Step 7:

[0568] The device notifies the user of reminders sent from the server. The notifications the user receives are emotionally sensitive and include advice tailored to the user's progress in moving preparations. This helps the user to proceed with the moving process smoothly.

[0569] (Application Example 2)

[0570] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0571] Providing appropriate customer service based on customer emotions is difficult, and there is a need for means to realize services that align with the diverse emotional states of customers. Conventional technologies have been unable to accurately grasp customer emotions and provide services accordingly, limiting the improvement of customer satisfaction.

[0572] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0573] In this invention, the server includes a device for acquiring user information, a device for processing the acquired user information and generating a list of necessary processing steps, and a device for selecting the most suitable service provider based on the user's status. This enables real-time analysis of customer sentiment data and the provision of optimal services based on the results.

[0574] A "device for acquiring user information" is a device for acquiring basic information and emotional data from users, thereby enabling the collection of detailed data about the user.

[0575] A "device that processes acquired user information and generates a list of necessary processes" is a device that analyzes and processes information acquired from users and lists appropriate procedures and service processes tailored to each user.

[0576] A "device that selects the optimal provider based on the user's condition" is a device that takes into account the user's emotions and condition and has the function of selecting the most appropriate service provider for the user.

[0577] A "processing list visualization device" is a device that displays a list of generated procedures and service processes to the user in an easy-to-understand manner, playing a role in helping the user understand them.

[0578] A "device that suggests related products and services" is a device that recommends products and services suitable for a user based on the user's data and emotional state.

[0579] A "device that provides notifications as aftercare" is a device that provides users with important information and reminders at an appropriate time after they have used a service.

[0580] A "device for acquiring user emotional data" is a device that collects a user's emotional state using devices such as cameras and microphones.

[0581] A "device for analyzing acquired emotional data" is a device that processes acquired emotional data and recognizes the user's current emotional state.

[0582] A "device that optimizes customer service methods and product suggestions based on emotional data" is a device that selects and suggests the most suitable customer service methods and products for a user based on analyzed emotional data.

[0583] This invention realizes a system in which a user, a terminal, and a server work together to provide optimal services based on emotions. When a user visits a store, smart glasses, which act as a terminal, are worn, and emotional data is acquired from the user's facial expressions and voice tone using a camera and microphone. The acquired data is transmitted to the server via an internet connection. The server analyzes the data using emotion recognition software, such as Microsoft Azure's facial recognition API, to understand the user's emotional state. Based on the analysis results, the server executes a program that selects the most suitable customer service method, related products, and services for the customer, and transmits the results to the terminal in real time. The terminal visualizes the customer service method and product suggestions based on the analysis results on its display. For example, a store employee wearing smart glasses may receive a notification such as "We recommend the new product promotion" when they recognize a customer's smile. An example of a prompt sentence using a generative AI model is, "Please suggest the best product for a customer who wants to relax." In this way, the system of the present invention can understand the user's emotions in real time and improve customer satisfaction through optimal service suggestions.

[0584] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0585] Step 1:

[0586] A user enters a store, and a store employee wearing smart glasses (which act as a terminal) begins assisting the customer. The smart glasses acquire the user's facial expressions and voice data in real time using a camera and microphone. This acquired data serves as input and is processed in the next step.

[0587] Step 2:

[0588] The device transmits the acquired facial expression and voice data to a server in the cloud via the internet. The server uses Microsoft Azure's facial recognition API to analyze the emotional data. Here, data calculations are performed to recognize the user's emotional state by analyzing the facial expression data and voice tone. The user's emotional state data is output as a result of the analysis.

[0589] Step 3:

[0590] Based on the analysis results, the server generates prompt messages using a generative AI model. At this time, prompt messages such as "Please suggest the best products for a customer who wants to relax" are generated. Based on these prompt messages, the server lists the most suitable products and services for the customer.

[0591] Step 4:

[0592] The product and service suggestions selected by the server are transmitted to the terminal in real time. The terminal receives this information and displays it on the smart glasses' display. This display is the output, and specific customer service methods and product suggestions are provided to the store staff as instructions.

[0593] Step 5:

[0594] Based on the suggestions accepted by the user, the store clerk explains product details to the customer and provides related services. Specific actions by the clerk include bringing the product to the customer's table and demonstrating it. Further feedback is obtained on the spot through interaction with the customer, and additional information is resent from the terminal to the server as needed.

[0595] In this way, users, devices, and servers work together to provide optimal service suggestions tailored to customer emotions, thereby improving the user experience.

[0596] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0597] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0598] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0599] [Fourth Embodiment]

[0600] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0601] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0602] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0603] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0604] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0606] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0607] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0608] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0609] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0610] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0611] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0612] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0613] This invention is implemented as a system to support the moving process. Specifically, it is designed to streamline the procedures required for moving, allowing users to smoothly begin their new life in their new home. The system primarily functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and further suggest products and services necessary for a new life at the new address.

[0614] First, users enter their old and new addresses, planned moving date, and other moving requirements via their smartphone or computer. This information is then transmitted from the device to a server in the cloud via the internet. The information is encrypted and processed securely.

[0615] Next, the server analyzes the received information and lists all procedures related to the user's new address. This is based on a database of local laws and regulations and general moving procedures. This allows for centralized management of procedures that previously required individual research.

[0616] The server then selects moving companies that meet the user's requirements and generates proposals from multiple companies. This allows the user to choose the best company based on cost, ratings, and services offered. The selection results can be viewed on the terminal, and the user can proceed with the contract immediately.

[0617] Furthermore, the server recommends relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. This makes it easy for users to arrange for the items they need to start their new life in their new home.

[0618] Finally, the invention also includes an aftercare function: the server schedules deadlines for important procedures and sends reminder notifications to the user via the terminal. This function prevents users from missing procedures and allows them to start their new life in their new home with peace of mind.

[0619] As a concrete example, consider a user moving to Tokyo. When this user enters their planned moving date and new address, the server lists the necessary items such as "notifying Tokyo residents of their change of address," "contracting for gas and water," and "selecting an internet provider," and presents recommended vendors and service providers. In this way, the present invention is designed to allow users to centrally manage complex and sporadic procedures.

[0620] The following describes the processing flow.

[0621] Step 1:

[0622] The user accesses the relocation support system and enters their new address, planned moving date, and any other additional information into the on-screen form. The terminal receives the data entered by the user and prepares to send it to the database.

[0623] Step 2:

[0624] The terminal sends user information to the server. The data being transmitted is encoded to ensure the security of the transmission. The terminal waits for confirmation of receipt from the server.

[0625] Step 3:

[0626] The server analyzes the received user information. It retrieves legal regulations and resident procedures related to the new address from the database and automatically generates the necessary procedures for the user. This list includes procedures for changing addresses and contracting for utilities (electricity, gas, water, etc.).

[0627] Step 4:

[0628] The server searches its database for moving companies based on the user's requirements. It takes into account service details, costs, and past ratings to generate a list of the most suitable companies for the user.

[0629] Step 5:

[0630] The terminal displays a generated list of procedures and information on selecting a moving company to the user. Based on this information, the user can choose a moving company and begin scheduling the necessary tasks.

[0631] Step 6:

[0632] Based on the user's past purchase history and preference data, the server suggests products and services that may be needed at their new address. This includes purchasing furniture and selecting an internet plan.

[0633] Step 7:

[0634] The terminal displays a list of products and services suggested by the server to the user. The user can click on a product or service of interest on the terminal to view details.

[0635] Step 8:

[0636] The server sets reminders for important procedures. The terminal sends a reminder notification to the user at the specified date and time to help the user remember the procedure.

[0637] (Example 1)

[0638] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0639] The procedures and related tasks involved in moving are complex, making it difficult for users to research and manage them individually. Furthermore, the wide variety of moving companies and necessary goods and services at the new residence makes it challenging for users to make optimal choices. In addition, managing deadlines for necessary procedures is not easy. A system is needed to address these problems and enable users to move efficiently and smoothly.

[0640] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0641] In this invention, the server includes means for generating a list of necessary procedures using a generated AI model based on user information, means including an algorithm for selecting the most suitable service provider based on user conditions, and means for managing deadlines for important procedures and providing reminder notifications to the user. This enables the user to proceed with moving procedures efficiently. Furthermore, by selecting the most suitable service provider and recommending related products and services, the user can move smoothly.

[0642] "User information" refers to information necessary when moving, such as the user's old and new addresses, planned moving date, and any special requirements.

[0643] A "generative AI model" is an artificial intelligence system that uses pattern recognition and machine learning based on user information to generate an optimal list of procedures.

[0644] A "service provider" is a company or organization that provides services related to moving, and is selected based on the user's requirements.

[0645] An "algorithm" is a computational method that systematizes the procedures for selecting the most suitable vendor based on user conditions and past evaluations.

[0646] "Preference information" refers to data that indicates an individual user's preferences and needs, based on their past selection and purchase history.

[0647] A "reminder notification" is a notification sent from a server to inform the user in advance of procedures or necessary actions they should take.

[0648] An "interface" refers to the user interface or connection method that allows a user to interact with a system and exchange information through their device.

[0649] This invention is a system that allows users to efficiently manage the moving process and smoothly start their new life. The system primarily has functions to automatically generate relevant procedures based on user input information, recommend the most suitable moving company, and suggest necessary goods and services at the new address.

[0650] Users enter their old and new addresses and moving date from their smartphones or computers. This entered information is transmitted via the internet to a server in the cloud. The data is encrypted using the SSL / TLS protocol to ensure security.

[0651] The server uses a generative AI model to analyze information received from the user and automatically generate a list of necessary procedures. This model refers to a regional legal and regulatory database to cover all procedures required at the user's new address. It also selects multiple moving companies based on the user's specified conditions, and the AI ​​algorithm generates these suggestions. Furthermore, the server analyzes the user's preference data and past purchase history to recommend products and services that will enrich their life at their new address.

[0652] The system manages the user's procedure schedule and sends reminders via the terminal when deadlines for important procedures are approaching. This feature helps users prevent missing procedures.

[0653] As a concrete example, a user might enter a prompt such as, "I'm planning to move in March, and my new address is in Tokyo. Please tell me what procedures I need to follow." Based on this information, the server creates a list of necessary procedures, such as "Notifying Tokyo of your change of address," "Contracting for gas and water services," and "Selecting an internet provider," and recommends the most suitable companies and products. This allows the user to easily manage the complex procedures related to moving and efficiently begin their new life.

[0654] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0655] Step 1:

[0656] Users enter their old and new addresses, planned moving date, and any special requirements using a smartphone or computer. The entered information is sent from the device to a server in the cloud. The data is encrypted using the SSL / TLS protocol during transmission, ensuring its security. The input consists of the user profile and moving conditions, which form the basis for generating procedures in the next step.

[0657] Step 2:

[0658] The server uses a generating AI model to analyze the received user information and generate a list of necessary procedures. The AI ​​model references regional legal and regulatory databases to cover all legal and administrative procedures related to the new address. It performs data calculations to list each procedure based on the local rule map and the new address. The output is a generated list of procedures, comprehensively covering all relevant items.

[0659] Step 3:

[0660] The server executes an algorithm to select the most suitable moving company based on the user's specified conditions. This algorithm generates a list of candidate companies based on evaluation criteria including ratings, cost, and service scope. The algorithm matches company information with the user's conditions, and the AI ​​guides the optimal selection. The output of this process is a list of candidate companies.

[0661] Step 4:

[0662] The server uses a generative AI model to analyze user preference data and past purchase history to recommend products and services that will enrich life at the new address. The analysis uses a product database and the user's past purchase information to list the most suitable products based on the user's interests. The output is a list of recommended products and services.

[0663] Step 5:

[0664] The server manages critical deadlines at each step of the procedure and sends reminder notifications to the user via their device at the specified date and time. This ensures the user completes the procedure without missing deadlines. Notifications are sent via email or in-app notifications and include procedure details and deadline information. The output is the scheduled notification.

[0665] (Application Example 1)

[0666] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0667] Moving requires numerous procedures and gathering information about the new living environment. However, these tasks are often very complicated, and choosing restaurants around the new address presents challenges, such as information overload and the difficulty of making appropriate choices within the limited time immediately after moving.

[0668] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0669] In this invention, the server includes means for inputting user information, means for processing the input user information and generating a list of necessary procedures, means for selecting the most suitable service provider based on the user's conditions, means for visualizing the list of procedures, means for suggesting related products and services, means for acquiring restaurant information based on the user's location information at their new address and generating meal suggestions based on the user's preferences, and means for providing reminders as aftercare. This not only efficiently manages procedures related to moving but also enables appropriate meal choices around the new address.

[0670] "User information" refers to personal data that users provide in order to handle procedures related to moving and to begin life at their new address.

[0671] A "procedure list" is a list of various procedures required for a user to move, and is generated based on local laws and regulations and general procedures.

[0672] "Vendor selection" is the process of choosing the most suitable moving company based on the user's requirements.

[0673] "Visualization" refers to presenting the generated list of procedures in an easy-to-understand manner to the user, enabling them to visually understand the progress of the procedures and the necessary actions.

[0674] "Suggestions for related products and services" refers to recommending suitable products and services based on the user's preferences and past purchase history in order to enrich their life at their new address.

[0675] "Location information" refers to geographical information about the area where a user is newly moving, and is used to improve the accuracy of suggestions.

[0676] "Restaurant information" refers to data about restaurants located near the user's new address, and is acquired and used as part of the recommendation process.

[0677] "Meal suggestions" refer to presenting appropriate meal options at the user's new address based on their preferences, and are suggestions aimed at improving the comfort of their life.

[0678] A "reminder" is a notification method that alerts users to deadlines for important procedures and events related to moving.

[0679] The system that realizes this invention is server-centric and operates in conjunction with the user's terminal. The user uses a terminal such as a smartphone or computer to input information necessary for moving, such as old and new addresses and moving date. This information is encrypted and transmitted to the server via the internet. Upon receiving this information, the server generates a list of necessary procedures based on local legal regulations and a database of general moving procedures.

[0680] The server also selects the most suitable moving company based on the user's requirements. It generates proposals from multiple companies and makes them available for review on the user's device. This allows the user to choose the best company while comparing costs, services, and other factors. Next, the server collects restaurant information based on the user's location to generate meal suggestions that reflect past preferences, in order to help the user choose meals related to their new address. This uses external data sources such as restaurant information APIs.

[0681] Furthermore, the server suggests relevant products and services to enrich life at the new address, based on the user's preference data and past purchase history. It also includes an after-care function that sends reminder notifications for important deadlines and events. Notifications are sent via SMS and email using the Twilio API, among others.

[0682] As a concrete example, let's assume a user is moving to Tokyo. The user enters their new address, and the server visualizes procedures such as "notifying the authorities of their change of address" and "contracting for water and gas." Furthermore, it suggests delivery information for popular nearby restaurants and restaurants related to the user's favorites.

[0683] An example of a prompt for the generating AI model is, "Please suggest some recommended restaurants and eateries for when I move to my new address {address}." This mechanism aims to comprehensively improve the user experience when moving.

[0684] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0685] Step 1:

[0686] The user enters their old and new addresses and moving date using their device. The entered information is encrypted and sent to the server via the internet. The system retrieves address information as input data and processes it to transfer securely.

[0687] Step 2:

[0688] The server uses the received user information to refer to a database of regional laws, regulations, and general procedures, and generates a list of necessary procedures. At this stage, data retrieval and listing are performed using the entered address information to obtain the procedure list output.

[0689] Step 3:

[0690] The server selects multiple moving companies based on the user's criteria and generates proposals from each company. Based on the conditions pre-set by the user, the server executes a company selection algorithm and generates a list of companies. This information is formatted for viewing on the terminal.

[0691] Step 4:

[0692] The server collects local restaurant data via an API based on the user's new address and generates meal suggestions by combining it with the user's preference data. Specifically, it performs data mining using location data as input to obtain output in the form of restaurant information and recommended menu items.

[0693] Step 5:

[0694] The server analyzes user preference data and past purchase history to suggest products and services relevant to life at the new address. It uses preference data as input to perform data calculations and apply product recommendation algorithms to output a list of related products.

[0695] Step 6:

[0696] The server generates reminders for important procedures and events, and notifies users via SMS or email using the Twilio API. The reminder generation process utilizes a scheduling function that takes schedule information and due date data as input and generates a notification message as output.

[0697] An example of a prompt would be, "Please suggest some restaurants or eateries in my new address {address}."

[0698] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0699] This invention combines emotion recognition technology with a system that supports the user's moving process to provide optimized services tailored to the user's emotions. In addition to functions such as user information input, information processing, procedure list generation, contractor selection, service proposal, and aftercare, this system is equipped with an emotion engine that grasps the user's emotions in real time and adjusts service proposals based on emotional aspects.

[0700] First, the user inputs basic information via a device such as a smartphone or computer and provides emotional data through emotion sensors (such as a camera or voice input). The device then acquires this data and prepares to send it to the cloud.

[0701] The server analyzes emotional data sent by the user and uses an emotion engine to recognize the user's current emotional state. This allows for, for example, the suggestion of special services to reduce stress or advice on procedures that are sensitive to the user's emotions. For instance, it can suggest procedures to help a user who is feeling stressed to help them relax.

[0702] Furthermore, the server uses user sentiment data to tailor the selection of moving companies based on those sentiments. In particular, it offers users who are experiencing stress the option to prioritize the selection of fast and reputable companies.

[0703] The products and services suggested are also tailored to the user's emotional state through the emotional engine. For example, a user who is emotionally exhausted will be recommended furniture and services that prioritize comfort.

[0704] Furthermore, the server adjusts reminders to the optimal timing and content based on the user's emotions, and delivers them to the user through their device. This system allows users to perform important procedures at a time when they are most emotionally receptive.

[0705] As a concrete example, when a user moves during a busy period, the system uses an emotion engine to detect the user's stress level and recommends moving companies and methods to expedite the process. At the same time, to ensure the user can complete the move without undue stress, the reminders sent are concise, helpful, and contain minimal notifications to reduce the burden. In this way, the system aims to provide emotional support and reduce stress throughout the user's moving process.

[0706] The following describes the processing flow.

[0707] Step 1:

[0708] Users log in to the relocation support system using their devices and enter basic profile information and detailed information about their move. They also provide emotional data through sensors such as cameras and microphones.

[0709] Step 2:

[0710] The terminal collects user information entered and emotional data acquired from sensors in real time, and transmits this data to a server. The data is encoded to ensure security and is efficiently transferred.

[0711] Step 3:

[0712] The server analyzes the received user data, retrieves the procedures the user needs at their new address from the database, and generates a list of necessary procedures. At this time, emotional data is also analyzed to understand the user's current emotional state.

[0713] Step 4:

[0714] The server considers emotional data to create optimal procedures tailored to the user's emotional state and offers suggestions to help reduce stress. For example, if the user's emotional state is unstable, it will prioritize recommending simple and easy procedures.

[0715] Step 5:

[0716] The server lists the most suitable moving companies based on the user's conditions and emotional state. For example, it recommends highly-rated companies and those capable of quick response times to users who are experiencing stress.

[0717] Step 6:

[0718] The terminal displays a generated list of procedures and proposals from moving companies to the user. The user can review the proposals and proceed with the procedures with the company they are interested in.

[0719] Step 7:

[0720] The server uses an emotion engine to suggest the most relevant products and services to the user. These suggestions are adjusted based on the user's emotional state.

[0721] Step 8:

[0722] The device will notify the user of the aforementioned proposals, allowing the user to review the options before proceeding with a purchase or contract.

[0723] Step 9:

[0724] The server takes the user's emotional state into consideration and generates reminders appropriate to the timing of the procedure. The reminders are adjusted to the user's most receptive timing, notifying them of important procedures and deadlines.

[0725] Step 10:

[0726] The device notifies the user of the generated reminder. The user receives the notification and can complete the task according to their schedule.

[0727] (Example 2)

[0728] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0729] During the moving process, the inability to provide optimal service that reflects the user's emotional state can cause stress to the user. Furthermore, the progress of the procedures may not be aligned with the user's situation, leading to problems such as unnecessary time and effort being required. This results in challenges that prevent the moving process from proceeding efficiently and comfortably.

[0730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0731] In this invention, the server includes means for collecting user information and analyzing emotional data, means for adjusting a list of procedures based on the emotional analysis results, and means for selecting and providing the most suitable service provider or service according to the user's emotions. This makes it possible to provide services that are appropriate to the user's emotional state, optimize the entire moving process, and realize a less burdensome experience for the user.

[0732] "User information" refers to basic personal data and conditions related to moving, including address, date, and desired conditions for the new residence.

[0733] "Emotional data" refers to information that quantifies or shows the user's psychological state, including data such as facial expressions and voice tone obtained using a camera or microphone.

[0734] "Emotional analysis" refers to the process of evaluating a user's current psychological state based on collected emotional data and identifying emotional states such as stress and reassurance.

[0735] A "procedure list" is a compilation of various specific steps and necessary items related to moving, serving as a guide for users to efficiently prepare for their move.

[0736] "Related services" refer to additional tasks and support provided to ensure a smooth and comfortable moving process, and include services such as cleaning, packing, and transportation.

[0737] A "reminder" is a notification that informs the user of necessary actions at a desired time, a function designed to prevent them from missing important procedures.

[0738] A "generative AI model" refers to an artificial intelligence technology used for natural language processing and data analysis, specifically a model designed to support user sentiment analysis and the optimization of service recommendations.

[0739] A "prompt sentence" is a natural language sentence input to give instructions to a generative AI model, serving as a trigger to obtain specific information or perform specific processing.

[0740] This invention is a system that supports the moving process while taking user emotions into consideration, and begins with inputting user information and emotional data using a device such as a smartphone or computer.

[0741] Users enter basic information such as their address and moving date via their device. Simultaneously, sensors such as the device's built-in camera and microphone provide emotional data, including facial expressions and voice. For example, if a user is experiencing stress, this emotional data is collected through their facial expressions and tone of voice.

[0742] The device sends collected information and emotional data to the cloud. On this cloud, data analysis is performed using a generative AI model. An emotional analysis engine operates, analyzing the user's psychological state in real time. The generative AI model is also used in the process of optimizing the next steps and services for the user.

[0743] Based on the results of the user's emotion analysis, the server can suggest services that are sensitive to the user's feelings and select the most suitable moving company. For example, if the user is feeling anxious, it will prioritize suggesting appropriate support to provide reassurance. Furthermore, it will also suggest products and services that reflect these results, recommending furniture and services that take the user's emotional state into consideration.

[0744] Furthermore, the server adjusts reminders based on the user's mental state, notifying them of important procedures at the appropriate time. This feature is designed to make the relocation process a more comfortable experience for the user.

[0745] As a concrete example, the entire system will operate by inputting the following prompt message into the generating AI model.

[0746] "My emotional sensors are detecting a high level of stress regarding my move. Please suggest procedures that will allow me to proceed efficiently while remaining relaxed."

[0747] Thus, this system aims to provide a user-friendly, stress-free moving process by offering optimal service suggestions and adjustments based on emotional data.

[0748] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0749] Step 1:

[0750] The user enters basic information about their move through the device. This includes their address, desired moving date, and requirements for their new home. Simultaneously, the device's camera and microphone are used to collect emotional data such as the user's facial expressions and voice. The device preprocesses this entered information and emotional data, converting it into a format suitable for transmission.

[0751] Step 2:

[0752] The terminal sends pre-processed information to a server in the cloud. Here, the entered user information and sentiment data are transferred to the server. The terminal's role is to transmit this information securely while maintaining data integrity.

[0753] Step 3:

[0754] The server analyzes the received emotional data using an AI model generated on the cloud. The emotional analysis engine then runs to identify the user's emotional state. The emotional data received as input is classified into states such as "reassured," "anxious," and "stressed" through computational processing using artificial intelligence technology. These analysis results are then used to propose the most suitable services.

[0755] Step 4:

[0756] The server generates a list of procedures and related services that are appropriate for the user's psychological state based on the analysis results. The generating AI model optimizes these and outputs content that is tailored to the user. Specifically, it provides procedures to enhance a sense of security and suggests products that have a relaxing effect.

[0757] Step 5:

[0758] The server selects the most suitable service provider for the user based on the analyzed emotional data. For users experiencing stress, it prioritizes and recommends service providers that are fast and have a good reputation. This selection process takes the user's psychological state into consideration.

[0759] Step 6:

[0760] The server creates reminders based on the user's status and prepares to send notifications at the appropriate time. The content and timing of the reminders are adjusted based on the prompt messages. For example, a suggestion such as, "We recommend you get some rest tonight in preparation for tomorrow's cargo removal," might be made.

[0761] Step 7:

[0762] The device notifies the user of reminders sent from the server. The notifications the user receives are emotionally sensitive and include advice tailored to the user's progress in moving preparations. This helps the user to proceed with the moving process smoothly.

[0763] (Application Example 2)

[0764] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0765] Providing appropriate customer service based on customer emotions is difficult, and there is a need for means to realize services that align with the diverse emotional states of customers. Conventional technologies have been unable to accurately grasp customer emotions and provide services accordingly, limiting the improvement of customer satisfaction.

[0766] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0767] In this invention, the server includes a device for acquiring user information, a device for processing the acquired user information and generating a list of necessary processing steps, and a device for selecting the most suitable service provider based on the user's status. This enables real-time analysis of customer sentiment data and the provision of optimal services based on the results.

[0768] A "device for acquiring user information" is a device for acquiring basic information and emotional data from users, thereby enabling the collection of detailed data about the user.

[0769] A "device that processes acquired user information and generates a list of necessary processes" is a device that analyzes and processes information acquired from users and lists appropriate procedures and service processes tailored to each user.

[0770] A "device that selects the optimal provider based on the user's condition" is a device that takes into account the user's emotions and condition and has the function of selecting the most appropriate service provider for the user.

[0771] A "processing list visualization device" is a device that displays a list of generated procedures and service processes to the user in an easy-to-understand manner, playing a role in helping the user understand them.

[0772] A "device that suggests related products and services" is a device that recommends products and services suitable for a user based on the user's data and emotional state.

[0773] A "device that provides notifications as aftercare" is a device that provides users with important information and reminders at an appropriate time after they have used a service.

[0774] A "device for acquiring user emotional data" is a device that collects a user's emotional state using devices such as cameras and microphones.

[0775] A "device for analyzing acquired emotional data" is a device that processes acquired emotional data and recognizes the user's current emotional state.

[0776] A "device that optimizes customer service methods and product suggestions based on emotional data" is a device that selects and suggests the most suitable customer service methods and products for a user based on analyzed emotional data.

[0777] This invention realizes a system in which a user, a terminal, and a server work together to provide optimal services based on emotions. When a user visits a store, smart glasses, which act as a terminal, are worn, and emotional data is acquired from the user's facial expressions and voice tone using a camera and microphone. The acquired data is transmitted to the server via an internet connection. The server analyzes the data using emotion recognition software, such as Microsoft Azure's facial recognition API, to understand the user's emotional state. Based on the analysis results, the server executes a program that selects the most suitable customer service method, related products, and services for the customer, and transmits the results to the terminal in real time. The terminal visualizes the customer service method and product suggestions based on the analysis results on its display. For example, a store employee wearing smart glasses may receive a notification such as "We recommend the new product promotion" when they recognize a customer's smile. An example of a prompt sentence using a generative AI model is, "Please suggest the best product for a customer who wants to relax." In this way, the system of the present invention can understand the user's emotions in real time and improve customer satisfaction through optimal service suggestions.

[0778] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0779] Step 1:

[0780] A user enters a store, and a store employee wearing smart glasses (which act as a terminal) begins assisting the customer. The smart glasses acquire the user's facial expressions and voice data in real time using a camera and microphone. This acquired data serves as input and is processed in the next step.

[0781] Step 2:

[0782] The device transmits the acquired facial expression and voice data to a server in the cloud via the internet. The server uses Microsoft Azure's facial recognition API to analyze the emotional data. Here, data calculations are performed to recognize the user's emotional state by analyzing the facial expression data and voice tone. The user's emotional state data is output as a result of the analysis.

[0783] Step 3:

[0784] Based on the analysis results, the server generates prompt messages using a generative AI model. At this time, prompt messages such as "Please suggest the best products for a customer who wants to relax" are generated. Based on these prompt messages, the server lists the most suitable products and services for the customer.

[0785] Step 4:

[0786] The product and service suggestions selected by the server are transmitted to the terminal in real time. The terminal receives this information and displays it on the smart glasses' display. This display is the output, and specific customer service methods and product suggestions are provided to the store staff as instructions.

[0787] Step 5:

[0788] Based on the suggestions accepted by the user, the store clerk explains product details to the customer and provides related services. Specific actions by the clerk include bringing the product to the customer's table and demonstrating it. Further feedback is obtained on the spot through interaction with the customer, and additional information is resent from the terminal to the server as needed.

[0789] In this way, users, devices, and servers work together to provide optimal service suggestions tailored to customer emotions, thereby improving the user experience.

[0790] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0791] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0792] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0793] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0794] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0795] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0796] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0797] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0798] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0799] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0800] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0801] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0802] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0803] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0804] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0805] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0806] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0807] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0808] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0809] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0810] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0811] The following is further disclosed regarding the embodiments described above.

[0812] (Claim 1)

[0813] Means of entering user information,

[0814] A means for processing the entered user information and generating a list of necessary procedures,

[0815] A means of selecting the most suitable vendor based on user requirements,

[0816] A means of visualizing the procedure list,

[0817] Means of proposing related products and services,

[0818] As a means of providing reminders as aftercare,

[0819] A system that includes this.

[0820] (Claim 2)

[0821] The system according to claim 1, comprising means for generating a procedure list based on input information and taking location information into consideration.

[0822] (Claim 3)

[0823] The system according to claim 1, comprising means for suggesting relevant products and services using user preference data and past purchase history.

[0824] "Example 1"

[0825] (Claim 1)

[0826] A device for inputting user information,

[0827] A device that generates a list of necessary procedures using an AI model based on input user information,

[0828] A device including an algorithm for selecting the optimal vendor based on user conditions,

[0829] A device that visualizes a list of procedures and provides it through an interface,

[0830] A device that recommends relevant products and services based on user preference information and past history,

[0831] A device that manages deadlines for important procedures and provides users with reminder notifications as aftercare,

[0832] A system that includes this.

[0833] (Claim 2)

[0834] The system according to claim 1, which generates a list of procedures using a generative AI model, taking location information into consideration, based on the input information.

[0835] (Claim 3)

[0836] The system according to claim 1, which recommends relevant products and services using user preference data and past transaction history.

[0837] "Application Example 1"

[0838] (Claim 1)

[0839] Means of entering user information,

[0840] A means for processing the entered user information and generating a list of necessary procedures,

[0841] A means of selecting the most suitable vendor based on user requirements,

[0842] A means of visualizing the procedure list,

[0843] Means of proposing related products and services,

[0844] A means of obtaining restaurant information based on the user's location information at their new address and generating meal suggestions based on the user's preferences,

[0845] As a means of providing reminders as aftercare,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, comprising means for generating a procedure list and meal suggestions based on input information and taking location information into consideration.

[0849] (Claim 3)

[0850] The system according to claim 1, comprising means for suggesting relevant products and services using user preference data and past order history.

[0851] "Example 2 of combining an emotion engine"

[0852] (Claim 1)

[0853] A means of inputting user information and collecting sentiment data,

[0854] A means for processing collected user information and sentiment data and performing sentiment analysis,

[0855] A means of proposing the optimal procedure list and related services based on the analysis results,

[0856] A method for selecting a vendor that is attentive to the user's feelings,

[0857] A means of adjusting the proposed related products and services to match the emotional state,

[0858] A means of optimizing and delivering reminders according to the user's emotions,

[0859] A system that includes this.

[0860] (Claim 2)

[0861] The system according to claim 1, comprising means for generating a procedure list taking into account location information and sentiment analysis results.

[0862] (Claim 3)

[0863] The system according to claim 1, comprising means for suggesting relevant products and services using user emotional data and past emotional states.

[0864] "Application example 2 when combining with an emotional engine"

[0865] (Claim 1)

[0866] A device for acquiring user information,

[0867] A device that processes acquired user information and generates a list of necessary processing steps,

[0868] A device that selects the most suitable vendor based on the user's status,

[0869] A device for visualizing the list of processes,

[0870] A device for proposing related products and services,

[0871] A device that provides notifications as aftercare,

[0872] A device for acquiring user emotion data,

[0873] A device for analyzing acquired emotional data,

[0874] A device that optimizes customer service methods and product suggestions based on emotional data.

[0875] A system that includes this.

[0876] (Claim 2)

[0877] The system according to claim 1, comprising a device that generates a processing list taking location information into consideration based on acquired information.

[0878] (Claim 3)

[0879] The system according to claim 1, comprising a device that suggests relevant products and services using user preference data and past purchase history. [Explanation of Symbols]

[0880] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means of entering user information, A means for processing the entered user information and generating a list of necessary procedures, A means of selecting the most suitable vendor based on user requirements, A means of visualizing the procedure list, Means of proposing related products and services, As a means of providing reminders as aftercare, A system that includes this.

2. The system according to claim 1, comprising means for generating a procedure list based on input information and taking location information into consideration.

3. The system according to claim 1, comprising means for suggesting relevant products and services using user preference data and past purchase history.

Images