system

Abstract

We provide the system. [Solution] A means by which the user accesses an automated vending system for purchasing a mobile device, A means for displaying an interactive interface and allowing the user to select their desired mobile plan and model, A means of sending the selection to the server and providing the latest plan and model information, Means for collecting selected options and payment information, A means of verifying payment and inventory information and performing authentication, A means of ejecting the selected mobile device after authentication is complete, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional mobile terminal purchase system, users are often made to wait at a counter for a long time, which is a great stress especially for tourists and busy businesspersons. Also, since explanations of complex options and plans are insufficient, it is difficult for users to select an appropriate mobile plan. Furthermore, since the purchase procedure is complex, there are many users who hesitate to purchase. There is a demand for a system that can solve such problems and enable quick and easy purchase and receipt of a mobile terminal.

Means for Solving the Problems

[0005] This invention provides a system that allows users to easily select, purchase, and receive mobile devices using an automated vending system. It includes a means for users to access a vending machine and select their desired mobile plan and model through an interactive interface. Furthermore, it includes a server that provides the latest plan and inventory information, and a means for authenticating payment and inventory information. The system also includes a means for users to directly receive the mobile device from the vending machine. This system reduces waiting times at service counters and allows users to obtain their desired mobile device quickly and intuitively.

[0006] An "automated vending system for purchasing mobile devices" refers to a vending machine and related systems that allow users to select, purchase, and receive mobile devices using an interactive interface.

[0007] A "user" is a consumer who attempts to purchase a mobile device using an automated vending system.

[0008] An "interactive interface" is an interface that guides the user through the process by sequentially displaying questions and selection screens based on their actions.

[0009] A "mobile phone plan" refers to a plan that includes the pricing structure for calls and data communication that a user chooses to use their mobile device.

[0010] "Model" refers to the specific mobile device model selected by the user.

[0011] A "server" is a computer system in an automated vending system that is responsible for processing and managing data, as well as communicating with external systems.

[0012] "Selected information" refers to information about the mobile phone plan and model chosen by the user through the interactive interface.

[0013] "Latest plan and device information" refers to the most up-to-date data on currently available mobile phone plans and devices.

[0014] "Payment method" refers to the payment method a user uses to purchase a mobile device. Specifically, this includes credit cards, QR code payments, etc.

[0015] "Payment information" refers to detailed information about the payment based on the payment method selected by the user.

[0016] "Inventory information" refers to information about the quantity and status of mobile devices held by the automated vending system.

[0017] "Authentication" refers to the process of verifying a user's payment and inventory information to ensure that the transaction is legitimate.

[0018] "Means for dispensing mobile devices" refers to the mechanism by which a vending machine dispenses a selected mobile device and provides it to the user.

[0019] A "QR code" is a two-dimensional barcode containing the plan and device information that the user has selected online in advance.

[0020] "Pre-application" refers to the procedure where users select and pay for their mobile phone plan and device in advance via the internet. [Brief explanation of the drawing]

[0021] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4]It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0022] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0025] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0026] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0027] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0029] [First Embodiment]

[0030] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0031] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0034] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0037] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0041] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0042] The present invention is a system that allows users to easily select, purchase, and receive a mobile device using an interactive automated vending system. Detailed embodiments of the present invention are described below.

[0043] overview

[0044] This system consists of a vending machine for purchasing mobile devices, a server that communicates with the vending machine, and a user. The user accesses the vending machine and selects the desired mobile device plan and model through an interactive interface. The system then checks the inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[0045] System Configuration

[0046] Vending machine (terminal)

[0047] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a built-in QR code scanner.

[0048] server

[0049] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[0050] Program processing

[0051] Selection process

[0052] When a user accesses the vending machine, the terminal first displays a language selection screen. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired mobile plan and device.

[0053] Information transmission and provision

[0054] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends them back to the device. This allows the user to make selections based on the latest information in real time.

[0055] Payment and Authentication

[0056] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected mobile device.

[0057] Examples

[0058] Example 1: Purchase on the spot

[0059] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and the latest smartphone model. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[0060] Example 2: Pre-order and pick-up

[0061] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at a vending machine in the airport. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine.

[0062] Thus, the system of the present invention provides a means for users to quickly and conveniently obtain a mobile device in busy airports and areas with many tourists.

[0063] The following describes the processing flow.

[0064] System processing flow - Pattern (1)

[0065] Step 1:

[0066] The user accesses the vending machine (terminal).

[0067] The device displays the initial screen and begins the operation guide.

[0068] Step 2:

[0069] The user selects the language using the touch panel.

[0070] The device displays an interactive interface according to the selected language.

[0071] Step 3:

[0072] The device displays a list of multiple mobile plans and models.

[0073] The user selects their desired plan and model using the touch panel.

[0074] Step 4:

[0075] The device sends the selected information to the server.

[0076] Based on the information received by the server, it calculates and returns the latest plan and model details.

[0077] Step 5:

[0078] The device receives the latest plan and device information from the server and displays it to the user.

[0079] The user selects options on the screen (e.g., compensation plan, accessories).

[0080] Step 6:

[0081] The terminal sends the selected option to the server.

[0082] The server calculates the final amount based on the options and sends it back.

[0083] Step 7:

[0084] The user selects a payment method.

[0085] The device displays the payment method (credit card, QR code, etc.).

[0086] Step 8:

[0087] The user enters payment information or scans a QR code.

[0088] The device sends payment information to the server.

[0089] Step 9:

[0090] The server authenticates the payment information and notifies the terminal of the authentication result.

[0091] Depending on the payment method, the system communicates with the authentication authority to confirm the payment.

[0092] Step 10:

[0093] The device receives an authentication completion notification and displays it to the user.

[0094] The terminal activates its internal mechanism and ejects the selected mobile device.

[0095] Step 11:

[0096] The user receives the mobile device.

[0097] The terminal displays the transaction completion screen, and the transaction is finished.

[0098] System processing flow - Pattern (2)

[0099] Step 1:

[0100] Users access websites via the internet.

[0101] The server displays an interactive interface, prompting the user to select a plan and model.

[0102] Step 2:

[0103] Users select their desired plan and model through an interactive interface.

[0104] The server records the selection.

[0105] Step 3:

[0106] The server displays a final confirmation screen where the user selects options and payment methods.

[0107] The user completes the payment online.

[0108] Step 4:

[0109] The server authenticates the payment information, and the authentication is completed.

[0110] The server generates a QR code containing the pre-application details and sends it to the user (via email or download).

[0111] Step 5:

[0112] The user arrives in Japan and accesses a vending machine at the airport.

[0113] The device displays a QR code scanning screen.

[0114] Step 6:

[0115] The user holds the QR code up to the vending machine's scanner.

[0116] The device reads the QR code and sends it to the server.

[0117] Step 7:

[0118] The server performs authentication based on the QR code information and sends the result to the device.

[0119] The server checks the inventory information and sends a pickup instruction to the terminal.

[0120] Step 8:

[0121] The device displays the authentication result to the user.

[0122] The terminal activates its internal mechanism and ejects a pre-selected mobile device.

[0123] Step 9:

[0124] The user receives the mobile device.

[0125] The terminal displays the transaction completion screen, and the transaction is finished.

[0126] (Example 1)

[0127] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0128] In modern society, the need for quick and easy purchase of communication devices is increasing due to busy schedules and the sudden need for communication devices while traveling. However, the existing communication device purchase process is often complicated and time-consuming. Therefore, there is a need for a way for users to quickly and efficiently select, purchase, and receive communication devices.

[0129] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0130] In this invention, the server includes means for the user to access an automated vending system for purchasing a communication terminal, means for displaying an interactive interface and for the user to select a desired communication plan and terminal, means for transmitting the selection to the server and providing the latest plan and terminal information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, and means for dispensing the selected communication terminal after authentication is complete. This enables the user to easily and quickly select and purchase a communication terminal and receive it immediately.

[0131] A "communication terminal" refers to an electronic device used for sending and receiving information.

[0132] An "automated vending system" refers to a machine or device that allows users to purchase goods on a self-service basis.

[0133] An "interactive interface" refers to a user interface that a user uses to interact with a system.

[0134] A "communication plan" refers to a pricing plan offered by a communication service provider that includes data capacity, call time, and other details.

[0135] A "server" refers to a computer system used for processing selections, providing information, authentication, and other related tasks.

[0136] A "QR code" is a type of two-dimensional barcode that enables the quick and easy reading of information.

[0137] "Payment information" refers to payment-related data, including credit card information and QR codes used for purchases.

[0138] "Inventory information" refers to data that shows the inventory status of a product.

[0139] "Authentication" refers to the procedure for verifying that the information sent is accurate.

[0140] "Dispensing" refers to the act of taking the selected product from the vending machine.

[0141] This invention provides a system that allows users to quickly and easily select, purchase, and receive a communication terminal using an interactive automated vending system. This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user who utilizes it. Specific embodiments are described below.

[0142] overview

[0143] The user accesses the vending machine and selects their desired communication plan and device through an interactive interface. The system then verifies the stock and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[0144] System Configuration

[0145] Vending machine (terminal)

[0146] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, communication plan and device selection, option selection, payment information input, and a built-in QR code scanner.

[0147] server

[0148] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[0149] Selection process

[0150] When a user accesses the vending machine, a language selection screen is displayed first. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired communication plan and device.

[0151] Specific example:

[0152] The system starts when the user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and latest communication device. They then press the confirmation button on the touch panel.

[0153] Information transmission and provision

[0154] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends that information back to the device in real time.

[0155] Specific example:

[0156] When a user selects a 10GB data plan and the latest communication device, that information is sent from the device to the server. The server checks the inventory status, calculates the latest information, and sends it back to the device.

[0157] Payment and Authentication

[0158] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected communication terminal.

[0159] Specific example:

[0160] When a user makes a payment with a credit card, the payment information is sent from the terminal to the server. The server authenticates the card information and sends the result back to the terminal. Once authentication is complete, the terminal checks its inventory and ejects the selected communication terminal.

[0161] Example of a prompt

[0162] An example of a prompt to be input into the generating AI model is: "Follow the instructions on the touch panel, select Japanese, then choose a 10GB data plan and the latest model of communication device. After that, complete the payment with your credit card."

[0163] As described above, the system of the present invention provides a means for users to quickly and conveniently obtain communication terminals in busy airports and areas with many tourists.

[0164] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0165] Step 1:

[0166] When a user accesses the vending machine, the terminal displays a language selection screen. The user uses the touch panel to select their desired language.

[0167] Input: User language selection (touch panel input)

[0168] Output: Display of an interactive interface according to language selection.

[0169] Specific actions:

[0170] The user selects "Japanese" from the options displayed on the language selection screen and confirms their selection by tapping the "Confirm" button on the touch panel.

[0171] Step 2:

[0172] Once language selection is complete, the device will launch an interactive interface and display a screen for selecting a communication plan. The user will then select their desired communication plan using the touch panel.

[0173] Input: User selects communication plan (touch panel input)

[0174] Output: Information on the selected communication plan

[0175] Specific actions:

[0176] The user selects the "10GB Data Plan" from the multiple data plan options displayed and confirms their selection by tapping the "Next" button on the screen.

[0177] Step 3:

[0178] Next, the terminal displays a screen for selecting a communication terminal. The user selects their desired communication terminal using the touch panel.

[0179] Input: User selects communication device (touch panel input)

[0180] Output: Information about the selected communication terminal

[0181] Specific actions:

[0182] The user selects "Latest model communication device" from the multiple communication device options displayed and confirms their selection by tapping the "Next" button on the screen.

[0183] Step 4:

[0184] Once the user's selection is finalized, the device sends that information to the server. The server processes the received information and sends the updated plan and device details back to the device.

[0185] Input: User selections (communication plan and communication device)

[0186] Output: Latest plan and device details

[0187] Specific actions:

[0188] The device sends its selection information for a "10GB data plan" and a "latest model communication device" to the server. The server checks the availability and validity of the plan and sends the results back to the device.

[0189] Step 5:

[0190] The user proceeds to the payment screen and completes the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result.

[0191] Input: Payment information (credit card information or QR code scan)

[0192] Output: Authentication result

[0193] Specific actions:

[0194] The user swipes their credit card through the terminal's scanner, and the payment information is sent from the terminal to the server. The server authenticates the card information and sends the authentication result back to the terminal.

[0195] Step 6:

[0196] Once authentication is complete, the terminal checks its inventory and ejects the selected communication device.

[0197] Input: Authentication results and inventory information

[0198] Output: Discharge of the selected communication terminal

[0199] Specific actions:

[0200] After payment is authenticated, the terminal checks the inventory and dispenses the communication device selected by the user from the vending machine. The device is dispensed when the user presses the "Receive" button on the touch panel.

[0201] (Application Example 1)

[0202] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0203] Purchasing mobile devices, especially in physical stores, is a time-consuming and cumbersome process for users. Therefore, there is a need for a simple and efficient way to select, purchase, and receive mobile devices. Furthermore, it is necessary to support diverse languages ​​and payment methods, and the provision of real-time information and inventory checks are crucial.

[0204] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0205] In this invention, the server includes means for providing an application that is installed on a smartphone and interacts with vending machines in physical stores; means for issuing a QR code, authenticating the user in the vending system based on the plan and model information selected in advance, and notifying the user of the authentication result via the smartphone application; and means for providing an interactive interface that interacts with vending machines in physical stores via the smartphone application, equipped with a language selection function and a voice guidance function for user operation. This enables users to easily and efficiently select, purchase, and receive mobile devices.

[0206] An "automated vending system" is a system of machines that allows users to automatically select and purchase products (in this case, mobile devices).

[0207] An "interactive interface" is an interface that allows the user and the system to exchange information with each other while performing operations.

[0208] "Selected information" refers to information such as the user's desired mobile phone plan and model.

[0209] A "server" is a computer system that processes and manages data for the entire system and communicates with other devices.

[0210] "Latest plan and device information" refers to information on currently available mobile plans and mobile devices.

[0211] "Payment information" refers to the information necessary for payment that a user provides when purchasing a mobile device.

[0212] "Inventory information" refers to information about the remaining quantity of products (mobile devices) held by the system.

[0213] "Authentication" is the process of verifying that a user is a legitimate purchaser based on the information and payment details they provide.

[0214] "Dispensing" refers to the physical action of the vending machine handing over the mobile device selected by the user.

[0215] "Applications installed on smartphones" refer to software applications that run on mobile devices such as mobile phones.

[0216] A "vending machine installed in a physical store" refers to a vending system machine that is installed inside a physical store.

[0217] A "QR code" is a two-dimensional barcode that encodes information in a compact, readable format.

[0218] The "language selection function" is a feature that allows users to choose their operating language from multiple languages.

[0219] The "voice guidance function" is a feature that uses voice to guide users through operations and other information.

[0220] This invention is a system for users to easily select, purchase, and receive mobile devices. This objective is achieved using a smartphone application that interacts with vending machines in physical stores. The system primarily consists of the following means:

[0221] System Configuration

[0222] vending machine

[0223] The vending machines are installed in physical stores and feature touch panels for user operation. They also have built-in voice guidance and QR code scanners to assist with selection.

[0224] Smartphone applications

[0225] The smartphone application provides an interactive interface for users to select and purchase mobile devices and plans. This application includes a language selection function and supports multiple languages. Furthermore, the application interacts with a server to provide real-time updates.

[0226] Program processing details

[0227] Server Role

[0228] The server processes selections and payment information sent from the smartphone application to provide the latest plan and device information. The server also verifies payment and inventory information and performs authentication. The software used includes Swift (iOS), Kotlin (Android), Square API, and Stripe API.

[0229] Terminal role

[0230] The terminal (vending machine) receives input from the user and provides an interactive interface using a touch panel and voice guidance. It also authenticates the user using a QR code scanner and dispenses the selected mobile device.

[0231] User roles

[0232] The user launches a smartphone application, selects a language, and chooses their desired mobile device and plan. Once the selection is confirmed, the information is sent to the server and the payment process begins. After authentication is complete, the user can pick up their mobile device from a vending machine at a physical store.

[0233] Usage example

[0234] Example 1: Purchase at a shopping mall

[0235] The user launches a smartphone application at a cafe in the shopping mall and selects their language, mobile device, and plan. After completing the payment process, a QR code is issued. The user then goes to a vending machine located in the mall, scans the QR code, and the device is dispensed.

[0236] Example 2: Purchase at an electronics store in an urban area

[0237] Users select their device and plan in advance through an application from their home and complete the payment. A QR code is issued, and they proceed to a designated electronics store's vending machine. Authentication is performed by scanning the QR code, and the device is received on the spot.

[0238] Example of a prompt

[0239] The following are examples of prompts to input into the generating AI model.

[0240] Design a smartphone application for physical stores based on the following invention. The application should include language selection, mobile device plan selection, payment and authentication processes, and integration with vending machines. It should also use Swift (iOS) or Kotlin (Android), utilize the Square API and Stripe API for payment processing, and provide a natural language description of the generated program.

[0241] In this way, a system is realized that allows users to smoothly select, purchase, and receive mobile devices.

[0242] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0243] Step 1:

[0244] The user launches the smartphone application. A language selection screen appears, and the user selects their desired language. The selected language information is entered into the application, and as a result, the interactive interface begins.

[0245] Step 2:

[0246] The user selects their desired mobile device and plan. The application collects the user's selections and sends them to the server. These selections include the device model, data plan, and options. Based on this data, the server calculates the latest inventory and plan information and sends it back to the application.

[0247] Step 3:

[0248] The server processes the received selections and provides the latest plan and model information. This process uses database queries to retrieve inventory and the best pricing plan, and then sends that information to the application.

[0249] Step 4:

[0250] The user confirms their selections and proceeds to the payment screen. The application collects payment information and sends it to the server. This information may include credit card details or QR code payment details. The server processes this data and performs payment authentication. Payment authentication is performed using the Square API or Stripe API.

[0251] Step 5:

[0252] The server authenticates the payment information. First, it checks if the payment method is valid and sends the result to the application. If successful, a notification indicating that the payment has been completed is displayed to the application.

[0253] Step 6:

[0254] Once payment is confirmed, the server generates a QR code based on the authentication information and sends it to the application. The user then uses this code to retrieve their mobile device from a vending machine at a physical store.

[0255] Step 7:

[0256] The user approaches a vending machine installed in a physical store and scans a QR code generated by a smartphone application. The vending machine reads the QR code and sends the information to a server.

[0257] Step 8:

[0258] The server verifies that the QR code is valid and performs authentication. If successful, the server sends an instruction to the vending machine to eject the mobile device. The vending machine then physically ejects the device based on this instruction.

[0259] Step 9:

[0260] The user can retrieve their selected mobile device from the vending machine. After retrieval, a notification confirming receipt will appear on the smartphone application. This completes all steps in the process.

[0261] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0262] This invention is a system that combines an automated vending system that allows for easy selection, purchase, and receipt of mobile devices with an emotion engine that recognizes the user's emotions. Detailed embodiments of this invention are described below.

[0263] overview

[0264] This system consists of a vending machine (terminal) for purchasing mobile devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile plan and model, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the system checks inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[0265] System Configuration

[0266] Vending machine (terminal)

[0267] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a QR code scanner.

[0268] server

[0269] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[0270] Details of the emotion recognition function

[0271] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[0272] Program processing

[0273] Selection process

[0274] When a user accesses the vending machine, the terminal first displays a language selection screen, and the emotion engine begins capturing the user's facial expressions and voice tone. After the user selects a language, an interactive interface begins, allowing them to sequentially select their desired mobile plan and model. The emotion engine analyzes the user's emotional state and adjusts the guidance and options as needed.

[0275] Information transmission and provision

[0276] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. This allows the user to make selections based on the latest information in real time. Emotion engine data is also sent to the server for reference.

[0277] Payment and Authentication

[0278] After the selection is confirmed, the user proceeds with the payment procedure. The terminal collects payment information through a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. When the authentication is completed, the terminal checks the inventory information and discharges the selected mobile terminal.

[0279] Embodiment

[0280] Example 1: On-site Purchase

[0281] The user accesses a vending machine installed at the airport and touches the touch panel to start the system. The user selects English, and then selects the desired data communication plan and the latest smartphone model. During this time, the emotion engine analyzes the user's expression and voice, and if the user is confused, additional explanations are displayed. Subsequently, when payment is made with a credit card, the server authenticates the information and the selected smartphone is discharged from the vending machine.

[0282] Example 2: Advance Reservation and Pick-up

[0283] [[ID=Is19]] The user selects the desired plan and model on the website via the Internet from home and proceeds with the payment procedure. Subsequently, a QR code is issued. After arriving in Japan at the travel destination, when the user holds the QR code up to the vending machine at the airport, the emotion engine checks the user's state and sends the information to the server. The server authenticates the information and the previously selected mobile terminal is discharged from the vending machine.

[0284] In this way, the system of the present invention provides a means to further improve the user experience and obtain a mobile terminal quickly and intuitively by combining an emotion recognition function.

[0285] The following describes the processing flow.

[0286] System Processing Flow - Invention Combining an Emotion Engine

[0287] Step 1:

[0288] The user accesses the vending machine (terminal).

[0289] The device displays the initial screen and begins the operation guide.

[0290] Step 2:

[0291] The user selects the language using the touch panel.

[0292] The device displays an interactive interface according to the selected language.

[0293] The device activates its emotion engine and begins capturing the user's facial expressions and voice.

[0294] Step 3:

[0295] The device displays a list of multiple mobile plans and models.

[0296] The user selects their desired plan and model using the touch panel.

[0297] The emotion engine analyzes the user's emotional state while they are making a selection.

[0298] Step 4:

[0299] If the user is confused, the device will display additional explanations or support messages.

[0300] If the user is satisfied, proceed to the next process.

[0301] Step 5:

[0302] The device sends the selected information to the server.

[0303] Based on the information received by the server, it calculates and returns the latest plan and model details.

[0304] Step 6:

[0305] The terminal receives the latest plan and model information from the server and displays it to the user.

[0306] The user selects options (compensation plan, accessories, etc.) on the screen.

[0307] The emotion engine analyzes the user's emotional state again.

[0308] Step 7:

[0309] The terminal sends the selected options to the server.

[0310] The server calculates and returns the final amount based on the options.

[0311] Step 8:

[0312] The user selects a payment method.

[0313] The terminal displays payment methods (credit card, QR code, etc.).

[0314] Step 9:

[0315] The user enters payment information or scans a QR code.

[0316] The terminal sends the payment information to the server.

[0317] The emotion engine monitors the user's stress level.

[0318] Step 10:

[0319] The server authenticates the payment information and notifies the terminal of the authentication result. <00010​​​​

[0321] Step 11:

[0322] The device receives an authentication completion notification and displays it to the user.

[0323] The terminal activates its internal mechanism and ejects the selected mobile device.

[0324] The emotion engine performs a final check on user satisfaction.

[0325] Step 12:

[0326] The user receives the mobile device.

[0327] The terminal displays the transaction completion screen, and the transaction is finished.

[0328] The emotion engine collects user feedback and uses it to improve the system in the future.

[0329] Examples

[0330] Example 1: Purchase on the spot

[0331] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and tone of voice. If the user appears confused, the device displays detailed explanations to make the selection easier. Next, the user makes a payment with a credit card, and the server authenticates the information. Once authentication is complete, the selected smartphone is dispensed from the vending machine. The emotion engine measures the user's satisfaction and saves it as feedback.

[0332] Example 2: Pre-order and pick-up

[0333] The user selects their desired plan and device via a website from their home and completes the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at an airport vending machine. An emotion engine checks the user's state and sends the information to a server. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine. The emotion engine measures the user's satisfaction and stores it as feedback.

[0334] Thus, by combining the system of the present invention with emotion recognition functionality, it significantly improves the user experience and provides a means for obtaining a mobile device quickly and intuitively.

[0335] (Example 2)

[0336] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0337] Traditional vending systems often caused confusion and stress for users during the selection and purchase process, highlighting the need for improved user experience. Furthermore, they lacked flexible, interactive interfaces that considered user emotions, resulting in insufficient support for smooth purchase decision-making. Streamlining inventory and payment information verification and authentication processes was also required.

[0338] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's facial expressions and voice and dynamically adjusting the interface and guide content based on the emotional state, means for transmitting the selection content to the server and providing the latest plan and terminal information, and means for confirming payment information and inventory information and performing authentication. This reduces the confusion and stress felt by the user and allows the purchasing process to proceed smoothly through a flexible interactive interface. In addition, the inventory confirmation and payment authentication processes are streamlined, and it becomes possible to receive the communication terminal quickly.

[0339] A "user" is someone who uses the system to purchase a communication device.

[0340] "Communication terminals" is a general term for electronic devices with communication capabilities, such as mobile phones, smartphones, and tablets.

[0341] An "automated vending system" refers to an automated vending machine installed for users to purchase communication devices.

[0342] An "interactive interface" is an interface that allows users to communicate with the system in a two-way manner and proceed with making choices and performing actions.

[0343] A "communication plan" is a contract plan that defines the terms and conditions for using data communication and call services provided by a telecommunications carrier.

[0344] An "emotion engine" is software or hardware that analyzes a user's emotional state in real time from their facial expressions and voice.

[0345] A "server" is a central control unit that communicates with terminals to perform data processing and authentication.

[0346] "Selected information" refers to the communication plan and device information chosen by the user through the interactive interface.

[0347] "Latest plan and device information" refers to the latest information on communication plans and communication devices held by the server.

[0348] "Payment information" refers to credit card information and other payment method information that users provide for purchases.

[0349] "Inventory information" refers to information regarding the quantity and types of communication terminals stored within the automated vending system.

[0350] A "QR code" is a two-dimensional barcode issued to store information that a user has selected in advance.

[0351] This invention is a system that combines an automated vending system that allows for the easy selection, purchase, and receipt of communication terminals with an emotion engine that recognizes the user's emotions. A detailed embodiment of this system is described below.

[0352] overview

[0353] This system consists of a vending machine (terminal) for purchasing communication devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired communication plan and device, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the user confirms inventory and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[0354] System Configuration

[0355] Vending machine (terminal)

[0356] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selection. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, communication plan and device selection, option selection, payment information input, and a QR code scanner.

[0357] server

[0358] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[0359] Details of the emotion recognition function

[0360] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[0361] Hardware and software to be used

[0362] 1. Hardware:

[0363] Vending machine (touch panel, camera, microphone, credit card scanner, QR code reader)

[0364] Server (equipped with a high-performance processor and large memory capacity)

[0365] 2. Software:

[0366] Programming language: Python

[0367] Emotion recognition libraries: OpenCV, Dlib

[0368] Web framework: Flask

[0369] Specific example

[0370] Example 1: Purchase on the spot

[0371] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and voice, displaying additional explanations if they appear confused. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[0372] Example 2: Pre-order and pick-up

[0373] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in their destination country, the user scans the QR code at an airport vending machine. An emotional engine checks the user's status and sends the information to a server. The server authenticates the information, and the pre-selected communication device is dispensed from the vending machine.

[0374] Examples of prompts for generative AI models

[0375] "Please explain in detail how this system's vending machines recognize user emotions and how they adjust the service content based on those emotions."

[0376] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0377] Step 1: Language Selection

[0378] Input: The user touches the vending machine's touch panel.

[0379] Specific operation: The device displays a language selection screen. The user selects their desired language using the touch panel.

[0380] Output: The terminal retrieves the language information selected by the user.

[0381] Step 2: Capture and analyze emotional states

[0382] Input: The terminal receives a signal indicating that the user has begun to operate, which is a signal that the language selection is complete.

[0383] Specific operation: The device's camera and microphone begin capturing the user's facial expressions and voice tone in real time. The emotion engine analyzes the captured data to recognize the user's emotional state (e.g., stress, satisfaction, confusion).

[0384] Output: User emotional state data is generated as a result of analysis by the emotion engine.

[0385] Step 3: Select your desired communication plan and device.

[0386] Input: The device displays an interactive interface based on the user's emotional state data and language selection information.

[0387] Specific operation: The user operates the touch panel to confirm and select their desired communication plan and communication device. The device dynamically adjusts the displayed options based on the analysis results of the emotion engine.

[0388] Output: The communication plan and device information selected by the user are registered on the device.

[0389] Step 4: Submitting information and obtaining updates

[0390] Input: The terminal generates request data that includes the user's selected communication plan and terminal information.

[0391] Specific operation: The terminal sends request data to the server. After receiving the request data, the server retrieves the latest plan and terminal information from the database and returns the calculation result to the terminal.

[0392] Output: The latest information from the server is sent to the terminal and displayed to the user.

[0393] Step 5: Payment Procedure

[0394] Input: The terminal displays the latest information retrieved from the server and the user's selection information.

[0395] Specific operation: The user enters payment information using a credit card scanner or QR code reader. The terminal collects the payment information and sends it to the server.

[0396] Output: Payment information is sent to the server and the authentication process begins.

[0397] Step 6: Authentication and device ejection

[0398] Input: The server receives the payment information and completes the authentication process.

[0399] Specific operation: The server authenticates the payment information and notifies the terminal of the result. If authentication is successful, the terminal checks the inventory information and ejects the selected communication terminal.

[0400] Output: The authenticated communication device is ejected from the terminal and received by the user.

[0401] This allows users to purchase communication devices in a simple way using emotion recognition technology, and to have a smooth purchasing experience while reducing stress and confusion through an interactive interface.

[0402] (Application Example 2)

[0403] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0404] Conventional automated vending systems often cause users to feel confused or anxious when making a purchase, resulting in a less-than-smooth selection process. Furthermore, a lack of appropriate guidance and suggestions tailored to the user's emotional state hinders the optimization of the user experience. This invention aims to solve these problems and provide a means to improve the user interface.

[0405] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0406] In this invention, the server includes means for a user to access an automated vending system for purchasing a mobile device, means for displaying an interactive interface and for the user to select a desired mobile plan and model, means for transmitting the selection to the server and providing the latest plan and model information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, means for ejecting the selected mobile device after authentication is complete, and means equipped with an emotion recognition engine that recognizes the user's emotions and dynamically adjusts the dialogue and suggestions. This makes it possible to provide appropriate guidance and suggestions according to the user's emotional state, optimizing the user experience and facilitating a smooth selection process.

[0407] An "automated vending system" is a system that allows users to easily select, purchase, and receive mobile devices and other products.

[0408] An "interactive interface" is an interface that enables two-way interaction between the user and the system, and assists in presenting options and inputting information.

[0409] An "emotion recognition engine" is a software or hardware component that analyzes and recognizes emotions from a user's facial expressions and voice tone.

[0410] "Recognizing user emotions" is the process of extracting and accurately determining a user's emotional state (e.g., stress, satisfaction, confusion, etc.) from their facial expressions and voice.

[0411] A "server" is a computer system that processes user input information and provides necessary information and services.

[0412] "Mobile phone plan" refers to the pricing plan and contract details of the mobile phone service that a user chooses.

[0413] "Model" refers to the specific model or type of mobile phone selected by the user.

[0414] "Payment information" refers to information about credit card details and other payment methods that users need to purchase products.

[0415] "Authentication" is the process of verifying and approving whether the entered information is correct.

[0416] "Dispensing" refers to the act of providing the selected product to the user from the vending machine.

[0417] This invention is a system that combines an emotion recognition engine with an automated vending system that allows users to easily select, purchase, and receive mobile devices. Detailed embodiments of this invention are described below.

[0418] overview

[0419] This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile phone plan and model. The system incorporates an emotion recognition engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection, the system confirms inventory and payment method, and upon successful authentication, the mobile phone is dispensed from the vending machine.

[0420] System Configuration

[0421] Vending machine (terminal)

[0422] The device features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion recognition engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, mobile plan and model selection, option selection, payment information entry, and a built-in QR code scanner.

[0423] server

[0424] The server processes the selections and payment information sent from the device and provides the latest plan and device information based on that information. It also authenticates the payment and inventory information and sends the results to the device. Data from the emotion recognition engine is also sent to the server for reference.

[0425] Details of the emotion recognition function

[0426] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[0427] Program implementation

[0428] Hardware and software to be used

[0429] Smartphone: A device operated by a user.

[0430] Emotion recognition software: Affectiva, Microsoft Azure Emotion API, etc.

[0431] Server: A backend system for data processing (e.g., Amazon Web Services, Firebase).

[0432] Data processing and data calculation

[0433] The server performs the following actions:

[0434] 1. Input Data Acquisition: User facial expressions and voice tone are captured in real time via the device's camera and microphone. The libraries used are WebRTC and OpenCV.

[0435] 2. Emotion Recognition: Emotion recognition software (such as Affectiva or Azure Emotion API) is used to analyze the user's emotions from the captured data. The analysis results are obtained in JSON format.

[0436] 3. Adjusting the interactive interface: Dynamically adjust the UI / UX display based on the user's emotional state. For example, if the user is frustrated, simplify the ordering process; if they are satisfied, present promotional information.

[0437] 4. Backend Integration: Selection data and sentiment data are sent to the server for storage, and appropriate information is provided in real time. The technologies used are Firebase and AWS.

[0438] As a concrete example, consider a scenario where a user opens a food delivery app on their smartphone and begins placing an order. The camera and microphone analyze the user's emotions in real time, and if it determines that the user is frustrated, a message appears asking, "Do you need help completing your order quickly?" If the user answers "yes," the process switches to completing the order in the shortest possible steps.

[0439] Example of a prompt

[0440] Develop an app that recognizes user emotions using facial expressions and voice tone, and adjusts the food delivery ordering process accordingly. For example, if a user is frustrated, the app simplifies the ordering process and helps them complete their order quickly. If the user is satisfied, it provides promotional information or suggests new menu items. Technologies used should include Affectiva, Azure Emotion API, Firebase, and AWS.

[0441] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0442] Step 1:

[0443] The user accesses the vending machine (terminal). A screen for selecting a language appears on the terminal's touch panel. Based on the user's selection, the system proceeds to the next step. The user's actions on the touch panel serve as input, and the next screen is displayed as output.

[0444] Step 2:

[0445] The device begins capturing the user's facial expressions and voice in real time using its camera and microphone. This data is then analyzed using emotion recognition software (such as Affectiva or Azure Emotion API). The input consists of the user's facial expressions and voice data, and the output is the analysis result (e.g., emotional state such as stress, satisfaction, or confusion).

[0446] Step 3:

[0447] The interactive interface is initiated, and the user sequentially selects their mobile plan and device. Based on the analysis results of the emotion recognition engine, the UI / UX display is dynamically adjusted. For example, if the system determines that the user is confused, it will either provide a more detailed explanation or simplify the options. The input consists of the analysis results of the emotional state and the user's selections, and the output is the adjusted interface.

[0448] Step 4:

[0449] Once the user's selection is confirmed, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. The input is the user's selection, and the output is the latest plan and device information sent to the device.

[0450] Step 5:

[0451] The user initiates the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The input is payment information, and the output is the payment authentication result.

[0452] Step 6:

[0453] The server authenticates the payment information and notifies the terminal of the result. If authentication is complete, the terminal checks the inventory information and ejects the selected mobile device. The inputs are payment information and inventory information, and the outputs are the authentication result and the ejection of the product.

[0454] Step 7:

[0455] Once all selection and payment processes are complete, the device sends the final data from the emotion recognition engine to the server. The server uses this data to record the user's experience and use it to improve future services. The input is the final analysis result of the emotional state, and the output is recording it in the database.

[0456] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0457] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0458] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0459] [Second Embodiment]

[0460] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0461] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0462] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0463] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0464] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0465] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0466] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0467] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0468] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0469] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0470] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0471] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0472] The present invention is a system that allows users to easily select, purchase, and receive a mobile device using an interactive automated vending system. Detailed embodiments of the present invention are described below.

[0473] overview

[0474] This system consists of a vending machine for purchasing mobile devices, a server that communicates with the vending machine, and a user. The user accesses the vending machine and selects the desired mobile device plan and model through an interactive interface. The system then checks the inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[0475] System Configuration

[0476] Vending machine (terminal)

[0477] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a built-in QR code scanner.

[0478] server

[0479] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[0480] Program processing

[0481] Selection process

[0482] When a user accesses the vending machine, the terminal first displays a language selection screen. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired mobile plan and device.

[0483] Information transmission and provision

[0484] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends them back to the device. This allows the user to make selections based on the latest information in real time.

[0485] Payment and Authentication

[0486] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected mobile device.

[0487] Examples

[0488] Example 1: Purchase on the spot

[0489] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and the latest smartphone model. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[0490] Example 2: Pre-order and pick-up

[0491] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at a vending machine in the airport. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine.

[0492] Thus, the system of the present invention provides a means for users to quickly and conveniently obtain a mobile device in busy airports and areas with many tourists.

[0493] The following describes the processing flow.

[0494] System processing flow - Pattern (1)

[0495] Step 1:

[0496] The user accesses the vending machine (terminal).

[0497] The device displays the initial screen and begins the operation guide.

[0498] Step 2:

[0499] The user selects the language using the touch panel.

[0500] The device displays an interactive interface according to the selected language.

[0501] Step 3:

[0502] The device displays a list of multiple mobile plans and models.

[0503] The user selects their desired plan and model using the touch panel.

[0504] Step 4:

[0505] The device sends the selected information to the server.

[0506] Based on the information received by the server, it calculates and returns the latest plan and model details.

[0507] Step 5:

[0508] The device receives the latest plan and device information from the server and displays it to the user.

[0509] The user selects options on the screen (e.g., compensation plan, accessories).

[0510] Step 6:

[0511] The terminal sends the selected option to the server.

[0512] The server calculates the final amount based on the options and sends it back.

[0513] Step 7:

[0514] The user selects a payment method.

[0515] The device displays the payment method (credit card, QR code, etc.).

[0516] Step 8:

[0517] The user enters payment information or scans a QR code.

[0518] The device sends payment information to the server.

[0519] Step 9:

[0520] The server authenticates the payment information and notifies the terminal of the authentication result.

[0521] Depending on the payment method, the system communicates with the authentication authority to confirm the payment.

[0522] Step 10:

[0523] The device receives an authentication completion notification and displays it to the user.

[0524] The terminal activates its internal mechanism and ejects the selected mobile device.

[0525] Step 11:

[0526] The user receives the mobile device.

[0527] The terminal displays the transaction completion screen, and the transaction is finished.

[0528] System processing flow - Pattern (2)

[0529] Step 1:

[0530] Users access websites via the internet.

[0531] The server displays an interactive interface, prompting the user to select a plan and model.

[0532] Step 2:

[0533] Users select their desired plan and model through an interactive interface.

[0534] The server records the selection.

[0535] Step 3:

[0536] The server displays a final confirmation screen where the user selects options and payment methods.

[0537] The user completes the payment online.

[0538] Step 4:

[0539] The server authenticates the payment information, and the authentication is completed.

[0540] The server generates a QR code containing the pre-application details and sends it to the user (via email or download).

[0541] Step 5:

[0542] The user arrives in Japan and accesses a vending machine at the airport.

[0543] The device displays a QR code scanning screen.

[0544] Step 6:

[0545] The user holds the QR code up to the vending machine's scanner.

[0546] The device reads the QR code and sends it to the server.

[0547] Step 7:

[0548] The server performs authentication based on the QR code information and sends the result to the device.

[0549] The server checks the inventory information and sends a pickup instruction to the terminal.

[0550] Step 8:

[0551] The device displays the authentication result to the user.

[0552] The terminal activates its internal mechanism and ejects a pre-selected mobile device.

[0553] Step 9:

[0554] The user receives the mobile device.

[0555] The terminal displays the transaction completion screen, and the transaction is finished.

[0556] (Example 1)

[0557] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0558] In modern society, the need for quick and easy purchase of communication devices is increasing due to busy schedules and the sudden need for communication devices while traveling. However, the existing communication device purchase process is often complicated and time-consuming. Therefore, there is a need for a way for users to quickly and efficiently select, purchase, and receive communication devices.

[0559] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0560] In this invention, the server includes means for the user to access an automated vending system for purchasing a communication terminal, means for displaying an interactive interface and for the user to select a desired communication plan and terminal, means for transmitting the selection to the server and providing the latest plan and terminal information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, and means for dispensing the selected communication terminal after authentication is complete. This enables the user to easily and quickly select and purchase a communication terminal and receive it immediately.

[0561] A "communication terminal" refers to an electronic device used for sending and receiving information.

[0562] An "automated vending system" refers to a machine or device that allows users to purchase goods on a self-service basis.

[0563] An "interactive interface" refers to a user interface that a user uses to interact with a system.

[0564] A "communication plan" refers to a pricing plan offered by a communication service provider that includes data capacity, call time, and other details.

[0565] A "server" refers to a computer system used for processing selections, providing information, authentication, and other related tasks.

[0566] A "QR code" is a type of two-dimensional barcode that enables the quick and easy reading of information.

[0567] "Payment information" refers to payment-related data, including credit card information and QR codes used for purchases.

[0568] "Inventory information" refers to data that shows the inventory status of a product.

[0569] "Authentication" refers to the procedure for verifying that the information sent is accurate.

[0570] "Dispensing" refers to the act of taking the selected product from the vending machine.

[0571] This invention provides a system that allows users to quickly and easily select, purchase, and receive a communication terminal using an interactive automated vending system. This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user who utilizes it. Specific embodiments are described below.

[0572] overview

[0573] The user accesses the vending machine and selects their desired communication plan and device through an interactive interface. The system then verifies the stock and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[0574] System Configuration

[0575] Vending machine (terminal)

[0576] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, communication plan and device selection, option selection, payment information input, and a built-in QR code scanner.

[0577] server

[0578] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[0579] Selection process

[0580] When a user accesses the vending machine, a language selection screen is displayed first. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired communication plan and device.

[0581] Specific example:

[0582] The system starts when the user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and latest communication device. They then press the confirmation button on the touch panel.

[0583] Information transmission and provision

[0584] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends that information back to the device in real time.

[0585] Specific example:

[0586] When a user selects a 10GB data plan and the latest communication device, that information is sent from the device to the server. The server checks the inventory status, calculates the latest information, and sends it back to the device.

[0587] Payment and Authentication

[0588] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected communication terminal.

[0589] Specific example:

[0590] When a user makes a payment with a credit card, the payment information is sent from the terminal to the server. The server authenticates the card information and sends the result back to the terminal. Once authentication is complete, the terminal checks its inventory and ejects the selected communication terminal.

[0591] Example of a prompt

[0592] An example of a prompt to be input into the generating AI model is: "Follow the instructions on the touch panel, select Japanese, then choose a 10GB data plan and the latest model of communication device. After that, complete the payment with your credit card."

[0593] As described above, the system of the present invention provides a means for users to quickly and conveniently obtain communication terminals in busy airports and areas with many tourists.

[0594] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0595] Step 1:

[0596] When a user accesses the vending machine, the terminal displays a language selection screen. The user uses the touch panel to select their desired language.

[0597] Input: User language selection (touch panel input)

[0598] Output: Display of an interactive interface according to language selection.

[0599] Specific actions:

[0600] The user selects "Japanese" from the options displayed on the language selection screen and confirms their selection by tapping the "Confirm" button on the touch panel.

[0601] Step 2:

[0602] Once language selection is complete, the device will launch an interactive interface and display a screen for selecting a communication plan. The user will then select their desired communication plan using the touch panel.

[0603] Input: User selects communication plan (touch panel input)

[0604] Output: Information on the selected communication plan

[0605] Specific actions:

[0606] The user selects the "10GB Data Plan" from the multiple data plan options displayed and confirms their selection by tapping the "Next" button on the screen.

[0607] Step 3:

[0608] Next, the terminal displays a screen for selecting a communication terminal. The user selects their desired communication terminal using the touch panel.

[0609] Input: User selects communication device (touch panel input)

[0610] Output: Information about the selected communication terminal

[0611] Specific actions:

[0612] The user selects "Latest model communication device" from the multiple communication device options displayed and confirms their selection by tapping the "Next" button on the screen.

[0613] Step 4:

[0614] Once the user's selection is finalized, the device sends that information to the server. The server processes the received information and sends the updated plan and device details back to the device.

[0615] Input: User selections (communication plan and communication device)

[0616] Output: Latest plan and device details

[0617] Specific actions:

[0618] The device sends its selection information for a "10GB data plan" and a "latest model communication device" to the server. The server checks the availability and validity of the plan and sends the results back to the device.

[0619] Step 5:

[0620] The user proceeds to the payment screen and completes the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result.

[0621] Input: Payment information (credit card information or QR code scan)

[0622] Output: Authentication result

[0623] Specific actions:

[0624] The user swipes their credit card through the terminal's scanner, and the payment information is sent from the terminal to the server. The server authenticates the card information and sends the authentication result back to the terminal.

[0625] Step 6:

[0626] Once authentication is complete, the terminal checks its inventory and ejects the selected communication device.

[0627] Input: Authentication results and inventory information

[0628] Output: Discharge of the selected communication terminal

[0629] Specific actions:

[0630] After payment is authenticated, the terminal checks the inventory and dispenses the communication device selected by the user from the vending machine. The device is dispensed when the user presses the "Receive" button on the touch panel.

[0631] (Application Example 1)

[0632] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0633] Purchasing mobile devices, especially in physical stores, is a time-consuming and cumbersome process for users. Therefore, there is a need for a simple and efficient way to select, purchase, and receive mobile devices. Furthermore, it is necessary to support diverse languages ​​and payment methods, and the provision of real-time information and inventory checks are crucial.

[0634] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0635] In this invention, the server includes means for providing an application that is installed on a smartphone and interacts with vending machines in physical stores; means for issuing a QR code, authenticating the user in the vending system based on the plan and model information selected in advance, and notifying the user of the authentication result via the smartphone application; and means for providing an interactive interface that interacts with vending machines in physical stores via the smartphone application, equipped with a language selection function and a voice guidance function for user operation. This enables users to easily and efficiently select, purchase, and receive mobile devices.

[0636] An "automated vending system" is a system of machines that allows users to automatically select and purchase products (in this case, mobile devices).

[0637] An "interactive interface" is an interface that allows the user and the system to exchange information with each other while performing operations.

[0638] "Selected information" refers to information such as the user's desired mobile phone plan and model.

[0639] A "server" is a computer system that processes and manages data for the entire system and communicates with other devices.

[0640] "Latest plan and device information" refers to information on currently available mobile plans and mobile devices.

[0641] "Payment information" refers to the information necessary for payment that a user provides when purchasing a mobile device.

[0642] "Inventory information" refers to information about the remaining quantity of products (mobile devices) held by the system.

[0643] "Authentication" is the process of verifying that a user is a legitimate purchaser based on the information and payment details they provide.

[0644] "Dispensing" refers to the physical action of the vending machine handing over the mobile device selected by the user.

[0645] "Applications installed on smartphones" refer to software applications that run on mobile devices such as mobile phones.

[0646] A "vending machine installed in a physical store" refers to a vending system machine that is installed inside a physical store.

[0647] A "QR code" is a two-dimensional barcode that encodes information in a compact, readable format.

[0648] The "language selection function" is a feature that allows users to choose their operating language from multiple languages.

[0649] The "voice guidance function" is a feature that uses voice to guide users through operations and other information.

[0650] This invention is a system for users to easily select, purchase, and receive mobile devices. This objective is achieved using a smartphone application that interacts with vending machines in physical stores. The system primarily consists of the following means:

[0651] System Configuration

[0652] vending machine

[0653] The vending machines are installed in physical stores and feature touch panels for user operation. They also have built-in voice guidance and QR code scanners to assist with selection.

[0654] Smartphone applications

[0655] The smartphone application provides an interactive interface for users to select and purchase mobile devices and plans. This application includes a language selection function and supports multiple languages. Furthermore, the application interacts with a server to provide real-time updates.

[0656] Program processing details

[0657] Server Role

[0658] The server processes selections and payment information sent from the smartphone application to provide the latest plan and device information. The server also verifies payment and inventory information and performs authentication. The software used includes Swift (iOS), Kotlin (Android), Square API, and Stripe API.

[0659] Terminal role

[0660] The terminal (vending machine) receives input from the user and provides an interactive interface using a touch panel and voice guidance. It also authenticates the user using a QR code scanner and dispenses the selected mobile device.

[0661] User roles

[0662] The user launches a smartphone application, selects a language, and chooses their desired mobile device and plan. Once the selection is confirmed, the information is sent to the server and the payment process begins. After authentication is complete, the user can pick up their mobile device from a vending machine at a physical store.

[0663] Usage example

[0664] Example 1: Purchase at a shopping mall

[0665] The user launches a smartphone application at a cafe in the shopping mall and selects their language, mobile device, and plan. After completing the payment process, a QR code is issued. The user then goes to a vending machine located in the mall, scans the QR code, and the device is dispensed.

[0666] Example 2: Purchase at an electronics store in an urban area

[0667] Users select their device and plan in advance through an application from their home and complete the payment. A QR code is issued, and they proceed to a designated electronics store's vending machine. Authentication is performed by scanning the QR code, and the device is received on the spot.

[0668] Example of a prompt

[0669] The following are examples of prompts to input into the generating AI model.

[0670] Design a smartphone application for physical stores based on the following invention. The application should include language selection, mobile device plan selection, payment and authentication processes, and integration with vending machines. It should also use Swift (iOS) or Kotlin (Android), utilize the Square API and Stripe API for payment processing, and provide a natural language description of the generated program.

[0671] In this way, a system is realized that allows users to smoothly select, purchase, and receive mobile devices.

[0672] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0673] Step 1:

[0674] The user launches the smartphone application. A language selection screen appears, and the user selects their desired language. The selected language information is entered into the application, and as a result, the interactive interface begins.

[0675] Step 2:

[0676] The user selects their desired mobile device and plan. The application collects the user's selections and sends them to the server. These selections include the device model, data plan, and options. Based on this data, the server calculates the latest inventory and plan information and sends it back to the application.

[0677] Step 3:

[0678] The server processes the received selections and provides the latest plan and model information. This process uses database queries to retrieve inventory and the best pricing plan, and then sends that information to the application.

[0679] Step 4:

[0680] The user confirms their selections and proceeds to the payment screen. The application collects payment information and sends it to the server. This information may include credit card details or QR code payment details. The server processes this data and performs payment authentication. Payment authentication is performed using the Square API or Stripe API.

[0681] Step 5:

[0682] The server authenticates the payment information. First, it checks if the payment method is valid and sends the result to the application. If successful, a notification indicating that the payment has been completed is displayed to the application.

[0683] Step 6:

[0684] Once payment is confirmed, the server generates a QR code based on the authentication information and sends it to the application. The user then uses this code to retrieve their mobile device from a vending machine at a physical store.

[0685] Step 7:

[0686] The user approaches a vending machine installed in a physical store and scans a QR code generated by a smartphone application. The vending machine reads the QR code and sends the information to a server.

[0687] Step 8:

[0688] The server verifies that the QR code is valid and performs authentication. If successful, the server sends an instruction to the vending machine to eject the mobile device. The vending machine then physically ejects the device based on this instruction.

[0689] Step 9:

[0690] The user can retrieve their selected mobile device from the vending machine. After retrieval, a notification confirming receipt will appear on the smartphone application. This completes all steps in the process.

[0691] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0692] This invention is a system that combines an automated vending system that allows for easy selection, purchase, and receipt of mobile devices with an emotion engine that recognizes the user's emotions. Detailed embodiments of this invention are described below.

[0693] overview

[0694] This system consists of a vending machine (terminal) for purchasing mobile devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile plan and model, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the system checks inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[0695] System Configuration

[0696] Vending machine (terminal)

[0697] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a QR code scanner.

[0698] server

[0699] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[0700] Details of the emotion recognition function

[0701] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[0702] Program processing

[0703] Selection process

[0704] When a user accesses the vending machine, the terminal first displays a language selection screen, and the emotion engine begins capturing the user's facial expressions and voice tone. After the user selects a language, an interactive interface begins, allowing them to sequentially select their desired mobile plan and model. The emotion engine analyzes the user's emotional state and adjusts the guidance and options as needed.

[0705] Information transmission and provision

[0706] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. This allows the user to make selections based on the latest information in real time. Emotion engine data is also sent to the server for reference.

[0707] Payment and Authentication

[0708] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected mobile device.

[0709] Examples

[0710] Example 1: Purchase on the spot

[0711] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and voice, displaying additional explanations if they appear confused. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[0712] Example 2: Pre-order and pick-up

[0713] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at a vending machine in the airport. An emotional engine checks the user's status and sends the information to a server. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine.

[0714] Thus, by combining the system of the present invention with emotion recognition functionality, it further enhances the user experience and provides a means for obtaining a mobile device quickly and intuitively.

[0715] The following describes the processing flow.

[0716] System processing flow - an invention combining an emotion engine

[0717] Step 1:

[0718] The user accesses the vending machine (terminal).

[0719] The device displays the initial screen and begins the operation guide.

[0720] Step 2:

[0721] The user selects the language using the touch panel.

[0722] The device displays an interactive interface according to the selected language.

[0723] The device activates its emotion engine and begins capturing the user's facial expressions and voice.

[0724] Step 3:

[0725] The device displays a list of multiple mobile plans and models.

[0726] The user selects their desired plan and model using the touch panel.

[0727] The emotion engine analyzes the user's emotional state while they are making a selection.

[0728] Step 4:

[0729] If the user is confused, the device will display additional explanations or support messages.

[0730] If the user is satisfied, proceed to the next process.

[0731] Step 5:

[0732] The device sends the selected information to the server.

[0733] Based on the information received by the server, it calculates and returns the latest plan and model details.

[0734] Step 6:

[0735] The device receives the latest plan and device information from the server and displays it to the user.

[0736] The user selects options on the screen (e.g., compensation plan, accessories).

[0737] The emotion engine analyzes the user's emotional state again.

[0738] Step 7:

[0739] The terminal sends the selected option to the server.

[0740] The server calculates the final amount based on the options and sends it back.

[0741] Step 8:

[0742] The user selects a payment method.

[0743] The device displays the payment method (credit card, QR code, etc.).

[0744] Step 9:

[0745] The user enters payment information or scans a QR code.

[0746] The device sends payment information to the server.

[0747] The emotion engine monitors the user's stress level.

[0748] Step 10:

[0749] The server authenticates the payment information and notifies the terminal of the authentication result.

[0750] Depending on the payment method, the system communicates with the authentication authority to confirm the payment.

[0751] Step 11:

[0752] The device receives an authentication completion notification and displays it to the user.

[0753] The terminal activates its internal mechanism and ejects the selected mobile device.

[0754] The emotion engine performs a final check on user satisfaction.

[0755] Step 12:

[0756] The user receives the mobile device.

[0757] The terminal displays the transaction completion screen, and the transaction is finished.

[0758] The emotion engine collects user feedback and uses it to improve the system in the future.

[0759] Examples

[0760] Example 1: Purchase on the spot

[0761] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and tone of voice. If the user appears confused, the device displays detailed explanations to make the selection easier. Next, the user makes a payment with a credit card, and the server authenticates the information. Once authentication is complete, the selected smartphone is dispensed from the vending machine. The emotion engine measures the user's satisfaction and saves it as feedback.

[0762] Example 2: Pre-order and pick-up

[0763] The user selects their desired plan and device via a website from their home and completes the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at an airport vending machine. An emotion engine checks the user's state and sends the information to a server. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine. The emotion engine measures the user's satisfaction and stores it as feedback.

[0764] Thus, by combining the system of the present invention with emotion recognition functionality, it significantly improves the user experience and provides a means for obtaining a mobile device quickly and intuitively.

[0765] (Example 2)

[0766] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0767] Traditional vending systems often caused confusion and stress for users during the selection and purchase process, highlighting the need for improved user experience. Furthermore, they lacked flexible, interactive interfaces that considered user emotions, resulting in insufficient support for smooth purchase decision-making. Streamlining inventory and payment information verification and authentication processes was also required.

[0768] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's facial expressions and voice and dynamically adjusting the interface and guide content based on the emotional state, means for transmitting the selection content to the server and providing the latest plan and terminal information, and means for confirming payment information and inventory information and performing authentication. This reduces the confusion and stress felt by the user and allows the purchasing process to proceed smoothly through a flexible interactive interface. In addition, the inventory confirmation and payment authentication processes are streamlined, and it becomes possible to receive the communication terminal quickly.

[0769] A "user" is someone who uses the system to purchase a communication device.

[0770] "Communication terminals" is a general term for electronic devices with communication capabilities, such as mobile phones, smartphones, and tablets.

[0771] An "automated vending system" refers to an automated vending machine installed for users to purchase communication devices.

[0772] An "interactive interface" is an interface that allows users to communicate with the system in a two-way manner and proceed with making choices and performing actions.

[0773] A "communication plan" is a contract plan that defines the terms and conditions for using data communication and call services provided by a telecommunications carrier.

[0774] An "emotion engine" is software or hardware that analyzes a user's emotional state in real time from their facial expressions and voice.

[0775] A "server" is a central control unit that communicates with terminals to perform data processing and authentication.

[0776] "Selected information" refers to the communication plan and device information chosen by the user through the interactive interface.

[0777] "Latest plan and device information" refers to the latest information on communication plans and communication devices held by the server.

[0778] "Payment information" refers to credit card information and other payment method information that users provide for purchases.

[0779] "Inventory information" refers to information regarding the quantity and types of communication terminals stored within the automated vending system.

[0780] A "QR code" is a two-dimensional barcode issued to store information that a user has selected in advance.

[0781] This invention is a system that combines an automated vending system that allows for the easy selection, purchase, and receipt of communication terminals with an emotion engine that recognizes the user's emotions. A detailed embodiment of this system is described below.

[0782] overview

[0783] This system consists of a vending machine (terminal) for purchasing communication devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired communication plan and device, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the user confirms inventory and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[0784] System Configuration

[0785] Vending machine (terminal)

[0786] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selection. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, communication plan and device selection, option selection, payment information input, and a QR code scanner.

[0787] server

[0788] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[0789] Details of the emotion recognition function

[0790] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[0791] Hardware and software to be used

[0792] 1. Hardware:

[0793] Vending machine (touch panel, camera, microphone, credit card scanner, QR code reader)

[0794] Server (equipped with a high-performance processor and large memory capacity)

[0795] 2. Software:

[0796] Programming language: Python

[0797] Emotion recognition libraries: OpenCV, Dlib

[0798] Web framework: Flask

[0799] Specific example

[0800] Example 1: Purchase on the spot

[0801] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and voice, displaying additional explanations if they appear confused. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[0802] Example 2: Pre-order and pick-up

[0803] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in their destination country, the user scans the QR code at an airport vending machine. An emotional engine checks the user's status and sends the information to a server. The server authenticates the information, and the pre-selected communication device is dispensed from the vending machine.

[0804] Examples of prompts for generative AI models

[0805] "Please explain in detail how this system's vending machines recognize user emotions and how they adjust the service content based on those emotions."

[0806] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0807] Step 1: Language Selection

[0808] Input: The user touches the vending machine's touch panel.

[0809] Specific operation: The device displays a language selection screen. The user selects their desired language using the touch panel.

[0810] Output: The terminal retrieves the language information selected by the user.

[0811] Step 2: Capture and analyze emotional states

[0812] Input: The terminal receives a signal indicating that the user has begun to operate, which is a signal that the language selection is complete.

[0813] Specific operation: The device's camera and microphone begin capturing the user's facial expressions and voice tone in real time. The emotion engine analyzes the captured data to recognize the user's emotional state (e.g., stress, satisfaction, confusion).

[0814] Output: User emotional state data is generated as a result of analysis by the emotion engine.

[0815] Step 3: Select your desired communication plan and device.

[0816] Input: The device displays an interactive interface based on the user's emotional state data and language selection information.

[0817] Specific operation: The user operates the touch panel to confirm and select their desired communication plan and communication device. The device dynamically adjusts the displayed options based on the analysis results of the emotion engine.

[0818] Output: The communication plan and device information selected by the user are registered on the device.

[0819] Step 4: Submitting information and obtaining updates

[0820] Input: The terminal generates request data that includes the user's selected communication plan and terminal information.

[0821] Specific operation: The terminal sends request data to the server. After receiving the request data, the server retrieves the latest plan and terminal information from the database and returns the calculation result to the terminal.

[0822] Output: The latest information from the server is sent to the terminal and displayed to the user.

[0823] Step 5: Payment Procedure

[0824] Input: The terminal displays the latest information retrieved from the server and the user's selection information.

[0825] Specific operation: The user enters payment information using a credit card scanner or QR code reader. The terminal collects the payment information and sends it to the server.

[0826] Output: Payment information is sent to the server and the authentication process begins.

[0827] Step 6: Authentication and device ejection

[0828] Input: The server receives the payment information and completes the authentication process.

[0829] Specific operation: The server authenticates the payment information and notifies the terminal of the result. If authentication is successful, the terminal checks the inventory information and ejects the selected communication terminal.

[0830] Output: The authenticated communication device is ejected from the terminal and received by the user.

[0831] This allows users to purchase communication devices in a simple way using emotion recognition technology, and to have a smooth purchasing experience while reducing stress and confusion through an interactive interface.

[0832] (Application Example 2)

[0833] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0834] Conventional automated vending systems often cause users to feel confused or anxious when making a purchase, resulting in a less-than-smooth selection process. Furthermore, a lack of appropriate guidance and suggestions tailored to the user's emotional state hinders the optimization of the user experience. This invention aims to solve these problems and provide a means to improve the user interface.

[0835] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0836] In this invention, the server includes means for a user to access an automated vending system for purchasing a mobile device, means for displaying an interactive interface and for the user to select a desired mobile plan and model, means for transmitting the selection to the server and providing the latest plan and model information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, means for ejecting the selected mobile device after authentication is complete, and means equipped with an emotion recognition engine that recognizes the user's emotions and dynamically adjusts the dialogue and suggestions. This makes it possible to provide appropriate guidance and suggestions according to the user's emotional state, optimizing the user experience and facilitating a smooth selection process.

[0837] An "automated vending system" is a system that allows users to easily select, purchase, and receive mobile devices and other products.

[0838] An "interactive interface" is an interface that enables two-way interaction between the user and the system, and assists in presenting options and inputting information.

[0839] An "emotion recognition engine" is a software or hardware component that analyzes and recognizes emotions from a user's facial expressions and voice tone.

[0840] "Recognizing user emotions" is the process of extracting and accurately determining a user's emotional state (e.g., stress, satisfaction, confusion, etc.) from their facial expressions and voice.

[0841] A "server" is a computer system that processes user input information and provides necessary information and services.

[0842] "Mobile phone plan" refers to the pricing plan and contract details of the mobile phone service that a user chooses.

[0843] "Model" refers to the specific model or type of mobile phone selected by the user.

[0844] "Payment information" refers to information about credit card details and other payment methods that users need to purchase products.

[0845] "Authentication" is the process of verifying and approving whether the entered information is correct.

[0846] "Dispensing" refers to the act of providing the selected product to the user from the vending machine.

[0847] This invention is a system that combines an emotion recognition engine with an automated vending system that allows users to easily select, purchase, and receive mobile devices. Detailed embodiments of this invention are described below.

[0848] overview

[0849] This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile phone plan and model. The system incorporates an emotion recognition engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection, the system confirms inventory and payment method, and upon successful authentication, the mobile phone is dispensed from the vending machine.

[0850] System Configuration

[0851] Vending machine (terminal)

[0852] The device features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion recognition engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, mobile plan and model selection, option selection, payment information entry, and a built-in QR code scanner.

[0853] server

[0854] The server processes the selections and payment information sent from the device and provides the latest plan and device information based on that information. It also authenticates the payment and inventory information and sends the results to the device. Data from the emotion recognition engine is also sent to the server for reference.

[0855] Details of the emotion recognition function

[0856] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[0857] Program implementation

[0858] Hardware and software to be used

[0859] Smartphone: A device operated by a user.

[0860] Emotion recognition software: Affectiva, Microsoft Azure Emotion API, etc.

[0861] Server: A backend system for data processing (e.g., Amazon Web Services, Firebase).

[0862] Data processing and data calculation

[0863] The server performs the following actions:

[0864] 1. Input Data Acquisition: User facial expressions and voice tone are captured in real time via the device's camera and microphone. The libraries used are WebRTC and OpenCV.

[0865] 2. Emotion Recognition: Emotion recognition software (such as Affectiva or Azure Emotion API) is used to analyze the user's emotions from the captured data. The analysis results are obtained in JSON format.

[0866] 3. Adjusting the interactive interface: Dynamically adjust the UI / UX display based on the user's emotional state. For example, if the user is frustrated, simplify the ordering process; if they are satisfied, present promotional information.

[0867] 4. Backend Integration: Selection data and sentiment data are sent to the server for storage, and appropriate information is provided in real time. The technologies used are Firebase and AWS.

[0868] As a concrete example, consider a scenario where a user opens a food delivery app on their smartphone and begins placing an order. The camera and microphone analyze the user's emotions in real time, and if it determines that the user is frustrated, a message appears asking, "Do you need help completing your order quickly?" If the user answers "yes," the process switches to completing the order in the shortest possible steps.

[0869] Example of a prompt

[0870] Develop an app that recognizes user emotions using facial expressions and voice tone, and adjusts the food delivery ordering process accordingly. For example, if a user is frustrated, the app simplifies the ordering process and helps them complete their order quickly. If the user is satisfied, it provides promotional information or suggests new menu items. Technologies used should include Affectiva, Azure Emotion API, Firebase, and AWS.

[0871] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0872] Step 1:

[0873] The user accesses the vending machine (terminal). A screen for selecting a language appears on the terminal's touch panel. Based on the user's selection, the system proceeds to the next step. The user's actions on the touch panel serve as input, and the next screen is displayed as output.

[0874] Step 2:

[0875] The device begins capturing the user's facial expressions and voice in real time using its camera and microphone. This data is then analyzed using emotion recognition software (such as Affectiva or Azure Emotion API). The input consists of the user's facial expressions and voice data, and the output is the analysis result (e.g., emotional state such as stress, satisfaction, or confusion).

[0876] Step 3:

[0877] The interactive interface is initiated, and the user sequentially selects their mobile plan and device. Based on the analysis results of the emotion recognition engine, the UI / UX display is dynamically adjusted. For example, if the system determines that the user is confused, it will either provide a more detailed explanation or simplify the options. The input consists of the analysis results of the emotional state and the user's selections, and the output is the adjusted interface.

[0878] Step 4:

[0879] Once the user's selection is confirmed, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. The input is the user's selection, and the output is the latest plan and device information sent to the device.

[0880] Step 5:

[0881] The user initiates the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The input is payment information, and the output is the payment authentication result.

[0882] Step 6:

[0883] The server authenticates the payment information and notifies the terminal of the result. If authentication is complete, the terminal checks the inventory information and ejects the selected mobile device. The inputs are payment information and inventory information, and the outputs are the authentication result and the ejection of the product.

[0884] Step 7:

[0885] Once all selection and payment processes are complete, the device sends the final data from the emotion recognition engine to the server. The server uses this data to record the user's experience and use it to improve future services. The input is the final analysis result of the emotional state, and the output is recording it in the database.

[0886] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0887] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0888] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0889] [Third Embodiment]

[0890] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0891] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0892] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0893] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0894] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0895] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0896] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0897] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0898] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0899] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0900] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0901] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0902] The present invention is a system that allows users to easily select, purchase, and receive a mobile device using an interactive automated vending system. Detailed embodiments of the present invention are described below.

[0903] overview

[0904] This system consists of a vending machine for purchasing mobile devices, a server that communicates with the vending machine, and a user. The user accesses the vending machine and selects the desired mobile device plan and model through an interactive interface. The system then checks the inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[0905] System Configuration

[0906] Vending machine (terminal)

[0907] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a built-in QR code scanner.

[0908] server

[0909] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[0910] Program processing

[0911] Selection process

[0912] When a user accesses the vending machine, the terminal first displays a language selection screen. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired mobile plan and device.

[0913] Information transmission and provision

[0914] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends them back to the device. This allows the user to make selections based on the latest information in real time.

[0915] Payment and Authentication

[0916] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected mobile device.

[0917] Examples

[0918] Example 1: Purchase on the spot

[0919] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and the latest smartphone model. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[0920] Example 2: Pre-order and pick-up

[0921] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at a vending machine in the airport. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine.

[0922] Thus, the system of the present invention provides a means for users to quickly and conveniently obtain a mobile device in busy airports and areas with many tourists.

[0923] The following describes the processing flow.

[0924] System processing flow - Pattern (1)

[0925] Step 1:

[0926] The user accesses the vending machine (terminal).

[0927] The device displays the initial screen and begins the operation guide.

[0928] Step 2:

[0929] The user selects the language using the touch panel.

[0930] The device displays an interactive interface according to the selected language.

[0931] Step 3:

[0932] The device displays a list of multiple mobile plans and models.

[0933] The user selects their desired plan and model using the touch panel.

[0934] Step 4:

[0935] The device sends the selected information to the server.

[0936] Based on the information received by the server, it calculates and returns the latest plan and model details.

[0937] Step 5:

[0938] The device receives the latest plan and device information from the server and displays it to the user.

[0939] The user selects options on the screen (e.g., compensation plan, accessories).

[0940] Step 6:

[0941] The terminal sends the selected option to the server.

[0942] The server calculates the final amount based on the options and sends it back.

[0943] Step 7:

[0944] The user selects a payment method.

[0945] The device displays the payment method (credit card, QR code, etc.).

[0946] Step 8:

[0947] The user enters payment information or scans a QR code.

[0948] The device sends payment information to the server.

[0949] Step 9:

[0950] The server authenticates the payment information and notifies the terminal of the authentication result.

[0951] Depending on the payment method, the system communicates with the authentication authority to confirm the payment.

[0952] Step 10:

[0953] The device receives an authentication completion notification and displays it to the user.

[0954] The terminal activates its internal mechanism and ejects the selected mobile device.

[0955] Step 11:

[0956] The user receives the mobile device.

[0957] The terminal displays the transaction completion screen, and the transaction is finished.

[0958] System processing flow - Pattern (2)

[0959] Step 1:

[0960] Users access websites via the internet.

[0961] The server displays an interactive interface, prompting the user to select a plan and model.

[0962] Step 2:

[0963] Users select their desired plan and model through an interactive interface.

[0964] The server records the selection.

[0965] Step 3:

[0966] The server displays a final confirmation screen where the user selects options and payment methods.

[0967] The user completes the payment online.

[0968] Step 4:

[0969] The server authenticates the payment information, and the authentication is completed.

[0970] The server generates a QR code containing the pre-application details and sends it to the user (via email or download).

[0971] Step 5:

[0972] The user arrives in Japan and accesses a vending machine at the airport.

[0973] The device displays a QR code scanning screen.

[0974] Step 6:

[0975] The user holds the QR code up to the vending machine's scanner.

[0976] The device reads the QR code and sends it to the server.

[0977] Step 7:

[0978] The server performs authentication based on the QR code information and sends the result to the device.

[0979] The server checks the inventory information and sends a pickup instruction to the terminal.

[0980] Step 8:

[0981] The device displays the authentication result to the user.

[0982] The terminal activates its internal mechanism and ejects a pre-selected mobile device.

[0983] Step 9:

[0984] The user receives the mobile device.

[0985] The terminal displays the transaction completion screen, and the transaction is finished.

[0986] (Example 1)

[0987] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0988] In modern society, the need for quick and easy purchase of communication devices is increasing due to busy schedules and the sudden need for communication devices while traveling. However, the existing communication device purchase process is often complicated and time-consuming. Therefore, there is a need for a way for users to quickly and efficiently select, purchase, and receive communication devices.

[0989] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0990] In this invention, the server includes means for the user to access an automated vending system for purchasing a communication terminal, means for displaying an interactive interface and for the user to select a desired communication plan and terminal, means for transmitting the selection to the server and providing the latest plan and terminal information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, and means for dispensing the selected communication terminal after authentication is complete. This enables the user to easily and quickly select and purchase a communication terminal and receive it immediately.

[0991] A "communication terminal" refers to an electronic device used for sending and receiving information.

[0992] An "automated vending system" refers to a machine or device that allows users to purchase goods on a self-service basis.

[0993] An "interactive interface" refers to a user interface that a user uses to interact with a system.

[0994] A "communication plan" refers to a pricing plan offered by a communication service provider that includes data capacity, call time, and other details.

[0995] A "server" refers to a computer system used for processing selections, providing information, authentication, and other related tasks.

[0996] A "QR code" is a type of two-dimensional barcode that enables the quick and easy reading of information.

[0997] "Payment information" refers to payment-related data, including credit card information and QR codes used for purchases.

[0998] "Inventory information" refers to data that shows the inventory status of a product.

[0999] "Authentication" refers to the procedure for verifying that the information sent is accurate.

[1000] "Dispensing" refers to the act of taking the selected product from the vending machine.

[1001] This invention provides a system that allows users to quickly and easily select, purchase, and receive a communication terminal using an interactive automated vending system. This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user who utilizes it. Specific embodiments are described below.

[1002] overview

[1003] The user accesses the vending machine and selects their desired communication plan and device through an interactive interface. The system then verifies the stock and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[1004] System Configuration

[1005] Vending machine (terminal)

[1006] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, communication plan and device selection, option selection, payment information input, and a built-in QR code scanner.

[1007] server

[1008] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[1009] Selection process

[1010] When a user accesses the vending machine, a language selection screen is displayed first. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired communication plan and device.

[1011] Specific example:

[1012] The system starts when the user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and latest communication device. They then press the confirmation button on the touch panel.

[1013] Information transmission and provision

[1014] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends that information back to the device in real time.

[1015] Specific example:

[1016] When a user selects a 10GB data plan and the latest communication device, that information is sent from the device to the server. The server checks the inventory status, calculates the latest information, and sends it back to the device.

[1017] Payment and Authentication

[1018] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected communication terminal.

[1019] Specific example:

[1020] When a user makes a payment with a credit card, the payment information is sent from the terminal to the server. The server authenticates the card information and sends the result back to the terminal. Once authentication is complete, the terminal checks its inventory and ejects the selected communication terminal.

[1021] Example of a prompt

[1022] An example of a prompt to be input into the generating AI model is: "Follow the instructions on the touch panel, select Japanese, then choose a 10GB data plan and the latest model of communication device. After that, complete the payment with your credit card."

[1023] As described above, the system of the present invention provides a means for users to quickly and conveniently obtain communication terminals in busy airports and areas with many tourists.

[1024] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1025] Step 1:

[1026] When a user accesses the vending machine, the terminal displays a language selection screen. The user uses the touch panel to select their desired language.

[1027] Input: User language selection (touch panel input)

[1028] Output: Display of an interactive interface according to language selection.

[1029] Specific actions:

[1030] The user selects "Japanese" from the options displayed on the language selection screen and confirms their selection by tapping the "Confirm" button on the touch panel.

[1031] Step 2:

[1032] Once language selection is complete, the device will launch an interactive interface and display a screen for selecting a communication plan. The user will then select their desired communication plan using the touch panel.

[1033] Input: User selects communication plan (touch panel input)

[1034] Output: Information on the selected communication plan

[1035] Specific actions:

[1036] The user selects the "10GB Data Plan" from the multiple data plan options displayed and confirms their selection by tapping the "Next" button on the screen.

[1037] Step 3:

[1038] Next, the terminal displays a screen for selecting a communication terminal. The user selects their desired communication terminal using the touch panel.

[1039] Input: User selects communication device (touch panel input)

[1040] Output: Information about the selected communication terminal

[1041] Specific actions:

[1042] The user selects "Latest model communication device" from the multiple communication device options displayed and confirms their selection by tapping the "Next" button on the screen.

[1043] Step 4:

[1044] Once the user's selection is finalized, the device sends that information to the server. The server processes the received information and sends the updated plan and device details back to the device.

[1045] Input: User selections (communication plan and communication device)

[1046] Output: Latest plan and device details

[1047] Specific actions:

[1048] The device sends its selection information for a "10GB data plan" and a "latest model communication device" to the server. The server checks the availability and validity of the plan and sends the results back to the device.

[1049] Step 5:

[1050] The user proceeds to the payment screen and completes the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result.

[1051] Input: Payment information (credit card information or QR code scan)

[1052] Output: Authentication result

[1053] Specific actions:

[1054] The user swipes their credit card through the terminal's scanner, and the payment information is sent from the terminal to the server. The server authenticates the card information and sends the authentication result back to the terminal.

[1055] Step 6:

[1056] Once authentication is complete, the terminal checks its inventory and ejects the selected communication device.

[1057] Input: Authentication results and inventory information

[1058] Output: Discharge of the selected communication terminal

[1059] Specific actions:

[1060] After payment is authenticated, the terminal checks the inventory and dispenses the communication device selected by the user from the vending machine. The device is dispensed when the user presses the "Receive" button on the touch panel.

[1061] (Application Example 1)

[1062] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1063] Purchasing mobile devices, especially in physical stores, is a time-consuming and cumbersome process for users. Therefore, there is a need for a simple and efficient way to select, purchase, and receive mobile devices. Furthermore, it is necessary to support diverse languages ​​and payment methods, and the provision of real-time information and inventory checks are crucial.

[1064] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1065] In this invention, the server includes means for providing an application that is installed on a smartphone and interacts with vending machines in physical stores; means for issuing a QR code, authenticating the user in the vending system based on the plan and model information selected in advance, and notifying the user of the authentication result via the smartphone application; and means for providing an interactive interface that interacts with vending machines in physical stores via the smartphone application, equipped with a language selection function and a voice guidance function for user operation. This enables users to easily and efficiently select, purchase, and receive mobile devices.

[1066] An "automated vending system" is a system of machines that allows users to automatically select and purchase products (in this case, mobile devices).

[1067] An "interactive interface" is an interface that allows the user and the system to exchange information with each other while performing operations.

[1068] "Selected information" refers to information such as the user's desired mobile phone plan and model.

[1069] A "server" is a computer system that processes and manages data for the entire system and communicates with other devices.

[1070] "Latest plan and device information" refers to information on currently available mobile plans and mobile devices.

[1071] "Payment information" refers to the information necessary for payment that a user provides when purchasing a mobile device.

[1072] "Inventory information" refers to information about the remaining quantity of products (mobile devices) held by the system.

[1073] "Authentication" is the process of verifying that a user is a legitimate purchaser based on the information and payment details they provide.

[1074] "Dispensing" refers to the physical action of the vending machine handing over the mobile device selected by the user.

[1075] "Applications installed on smartphones" refer to software applications that run on mobile devices such as mobile phones.

[1076] A "vending machine installed in a physical store" refers to a vending system machine that is installed inside a physical store.

[1077] A "QR code" is a two-dimensional barcode that encodes information in a compact, readable format.

[1078] The "language selection function" is a feature that allows users to choose their operating language from multiple languages.

[1079] The "voice guidance function" is a feature that uses voice to guide users through operations and other information.

[1080] This invention is a system for users to easily select, purchase, and receive mobile devices. This objective is achieved using a smartphone application that interacts with vending machines in physical stores. The system primarily consists of the following means:

[1081] System Configuration

[1082] vending machine

[1083] The vending machines are installed in physical stores and feature touch panels for user operation. They also have built-in voice guidance and QR code scanners to assist with selection.

[1084] Smartphone applications

[1085] The smartphone application provides an interactive interface for users to select and purchase mobile devices and plans. This application includes a language selection function and supports multiple languages. Furthermore, the application interacts with a server to provide real-time updates.

[1086] Program processing details

[1087] Server Role

[1088] The server processes selections and payment information sent from the smartphone application to provide the latest plan and device information. The server also verifies payment and inventory information and performs authentication. The software used includes Swift (iOS), Kotlin (Android), Square API, and Stripe API.

[1089] Terminal role

[1090] The terminal (vending machine) receives input from the user and provides an interactive interface using a touch panel and voice guidance. It also authenticates the user using a QR code scanner and dispenses the selected mobile device.

[1091] User roles

[1092] The user launches a smartphone application, selects a language, and chooses their desired mobile device and plan. Once the selection is confirmed, the information is sent to the server and the payment process begins. After authentication is complete, the user can pick up their mobile device from a vending machine at a physical store.

[1093] Usage example

[1094] Example 1: Purchase at a shopping mall

[1095] The user launches a smartphone application at a cafe in the shopping mall and selects their language, mobile device, and plan. After completing the payment process, a QR code is issued. The user then goes to a vending machine located in the mall, scans the QR code, and the device is dispensed.

[1096] Example 2: Purchase at an electronics store in an urban area

[1097] Users select their device and plan in advance through an application from their home and complete the payment. A QR code is issued, and they proceed to a designated electronics store's vending machine. Authentication is performed by scanning the QR code, and the device is received on the spot.

[1098] Example of a prompt

[1099] The following are examples of prompts to input into the generating AI model.

[1100] Design a smartphone application for physical stores based on the following invention. The application should include language selection, mobile device plan selection, payment and authentication processes, and integration with vending machines. It should also use Swift (iOS) or Kotlin (Android), utilize the Square API and Stripe API for payment processing, and provide a natural language description of the generated program.

[1101] In this way, a system is realized that allows users to smoothly select, purchase, and receive mobile devices.

[1102] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1103] Step 1:

[1104] The user launches the smartphone application. A language selection screen appears, and the user selects their desired language. The selected language information is entered into the application, and as a result, the interactive interface begins.

[1105] Step 2:

[1106] The user selects their desired mobile device and plan. The application collects the user's selections and sends them to the server. These selections include the device model, data plan, and options. Based on this data, the server calculates the latest inventory and plan information and sends it back to the application.

[1107] Step 3:

[1108] The server processes the received selections and provides the latest plan and model information. This process uses database queries to retrieve inventory and the best pricing plan, and then sends that information to the application.

[1109] Step 4:

[1110] The user confirms their selections and proceeds to the payment screen. The application collects payment information and sends it to the server. This information may include credit card details or QR code payment details. The server processes this data and performs payment authentication. Payment authentication is performed using the Square API or Stripe API.

[1111] Step 5:

[1112] The server authenticates the payment information. First, it checks if the payment method is valid and sends the result to the application. If successful, a notification indicating that the payment has been completed is displayed to the application.

[1113] Step 6:

[1114] Once payment is confirmed, the server generates a QR code based on the authentication information and sends it to the application. The user then uses this code to retrieve their mobile device from a vending machine at a physical store.

[1115] Step 7:

[1116] The user approaches a vending machine installed in a physical store and scans a QR code generated by a smartphone application. The vending machine reads the QR code and sends the information to a server.

[1117] Step 8:

[1118] The server verifies that the QR code is valid and performs authentication. If successful, the server sends an instruction to the vending machine to eject the mobile device. The vending machine then physically ejects the device based on this instruction.

[1119] Step 9:

[1120] The user can retrieve their selected mobile device from the vending machine. After retrieval, a notification confirming receipt will appear on the smartphone application. This completes all steps in the process.

[1121] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1122] This invention is a system that combines an automated vending system that allows for easy selection, purchase, and receipt of mobile devices with an emotion engine that recognizes the user's emotions. Detailed embodiments of this invention are described below.

[1123] overview

[1124] This system consists of a vending machine (terminal) for purchasing mobile devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile plan and model, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the system checks inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[1125] System Configuration

[1126] Vending machine (terminal)

[1127] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a QR code scanner.

[1128] server

[1129] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[1130] Details of the emotion recognition function

[1131] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[1132] Program processing

[1133] Selection process

[1134] When a user accesses the vending machine, the terminal first displays a language selection screen, and the emotion engine begins capturing the user's facial expressions and voice tone. After the user selects a language, an interactive interface begins, allowing them to sequentially select their desired mobile plan and model. The emotion engine analyzes the user's emotional state and adjusts the guidance and options as needed.

[1135] Information transmission and provision

[1136] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. This allows the user to make selections based on the latest information in real time. Emotion engine data is also sent to the server for reference.

[1137] Payment and Authentication

[1138] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected mobile device.

[1139] Examples

[1140] Example 1: Purchase on the spot

[1141] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and voice, displaying additional explanations if they appear confused. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[1142] Example 2: Pre-order and pick-up

[1143] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at a vending machine in the airport. An emotional engine checks the user's status and sends the information to a server. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine.

[1144] Thus, by combining the system of the present invention with emotion recognition functionality, it further enhances the user experience and provides a means for obtaining a mobile device quickly and intuitively.

[1145] The following describes the processing flow.

[1146] System processing flow - an invention combining an emotion engine

[1147] Step 1:

[1148] The user accesses the vending machine (terminal).

[1149] The device displays the initial screen and begins the operation guide.

[1150] Step 2:

[1151] The user selects the language using the touch panel.

[1152] The device displays an interactive interface according to the selected language.

[1153] The device activates its emotion engine and begins capturing the user's facial expressions and voice.

[1154] Step 3:

[1155] The device displays a list of multiple mobile plans and models.

[1156] The user selects their desired plan and model using the touch panel.

[1157] The emotion engine analyzes the user's emotional state while they are making a selection.

[1158] Step 4:

[1159] If the user is confused, the device will display additional explanations or support messages.

[1160] If the user is satisfied, proceed to the next process.

[1161] Step 5:

[1162] The device sends the selected information to the server.

[1163] Based on the information received by the server, it calculates and returns the latest plan and model details.

[1164] Step 6:

[1165] The device receives the latest plan and device information from the server and displays it to the user.

[1166] The user selects options on the screen (e.g., compensation plan, accessories).

[1167] The emotion engine analyzes the user's emotional state again.

[1168] Step 7:

[1169] The terminal sends the selected option to the server.

[1170] The server calculates the final amount based on the options and sends it back.

[1171] Step 8:

[1172] The user selects a payment method.

[1173] The device displays the payment method (credit card, QR code, etc.).

[1174] Step 9:

[1175] The user enters payment information or scans a QR code.

[1176] The device sends payment information to the server.

[1177] The emotion engine monitors the user's stress level.

[1178] Step 10:

[1179] The server authenticates the payment information and notifies the terminal of the authentication result.

[1180] Depending on the payment method, the system communicates with the authentication authority to confirm the payment.

[1181] Step 11:

[1182] The device receives an authentication completion notification and displays it to the user.

[1183] The terminal activates its internal mechanism and ejects the selected mobile device.

[1184] The emotion engine performs a final check on user satisfaction.

[1185] Step 12:

[1186] The user receives the mobile device.

[1187] The terminal displays the transaction completion screen, and the transaction is finished.

[1188] The emotion engine collects user feedback and uses it to improve the system in the future.

[1189] Examples

[1190] Example 1: Purchase on the spot

[1191] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and tone of voice. If the user appears confused, the device displays detailed explanations to make the selection easier. Next, the user makes a payment with a credit card, and the server authenticates the information. Once authentication is complete, the selected smartphone is dispensed from the vending machine. The emotion engine measures the user's satisfaction and saves it as feedback.

[1192] Example 2: Pre-order and pick-up

[1193] The user selects their desired plan and device via a website from their home and completes the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at an airport vending machine. An emotion engine checks the user's state and sends the information to a server. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine. The emotion engine measures the user's satisfaction and stores it as feedback.

[1194] Thus, by combining the system of the present invention with emotion recognition functionality, it significantly improves the user experience and provides a means for obtaining a mobile device quickly and intuitively.

[1195] (Example 2)

[1196] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1197] Traditional vending systems often caused confusion and stress for users during the selection and purchase process, highlighting the need for improved user experience. Furthermore, they lacked flexible, interactive interfaces that considered user emotions, resulting in insufficient support for smooth purchase decision-making. Streamlining inventory and payment information verification and authentication processes was also required.

[1198] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's facial expressions and voice and dynamically adjusting the interface and guide content based on the emotional state, means for transmitting the selection content to the server and providing the latest plan and terminal information, and means for confirming payment information and inventory information and performing authentication. This reduces the confusion and stress felt by the user and allows the purchasing process to proceed smoothly through a flexible interactive interface. In addition, the inventory confirmation and payment authentication processes are streamlined, and it becomes possible to receive the communication terminal quickly.

[1199] A "user" is someone who uses the system to purchase a communication device.

[1200] "Communication terminals" is a general term for electronic devices with communication capabilities, such as mobile phones, smartphones, and tablets.

[1201] An "automated vending system" refers to an automated vending machine installed for users to purchase communication devices.

[1202] An "interactive interface" is an interface that allows users to communicate with the system in a two-way manner and proceed with making choices and performing actions.

[1203] A "communication plan" is a contract plan that defines the terms and conditions for using data communication and call services provided by a telecommunications carrier.

[1204] An "emotion engine" is software or hardware that analyzes a user's emotional state in real time from their facial expressions and voice.

[1205] A "server" is a central control unit that communicates with terminals to perform data processing and authentication.

[1206] "Selected information" refers to the communication plan and device information chosen by the user through the interactive interface.

[1207] "Latest plan and device information" refers to the latest information on communication plans and communication devices held by the server.

[1208] "Payment information" refers to credit card information and other payment method information that users provide for purchases.

[1209] "Inventory information" refers to information regarding the quantity and types of communication terminals stored within the automated vending system.

[1210] A "QR code" is a two-dimensional barcode issued to store information that a user has selected in advance.

[1211] This invention is a system that combines an automated vending system that allows for the easy selection, purchase, and receipt of communication terminals with an emotion engine that recognizes the user's emotions. A detailed embodiment of this system is described below.

[1212] overview

[1213] This system consists of a vending machine (terminal) for purchasing communication devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired communication plan and device, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the user confirms inventory and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[1214] System Configuration

[1215] Vending machine (terminal)

[1216] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selection. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, communication plan and device selection, option selection, payment information input, and a QR code scanner.

[1217] server

[1218] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[1219] Details of the emotion recognition function

[1220] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[1221] Hardware and software to be used

[1222] 1. Hardware:

[1223] Vending machine (touch panel, camera, microphone, credit card scanner, QR code reader)

[1224] Server (equipped with a high-performance processor and large memory capacity)

[1225] 2. Software:

[1226] Programming language: Python

[1227] Emotion recognition libraries: OpenCV, Dlib

[1228] Web framework: Flask

[1229] Specific example

[1230] Example 1: Purchase on the spot

[1231] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and voice, displaying additional explanations if they appear confused. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[1232] Example 2: Pre-order and pick-up

[1233] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in their destination country, the user scans the QR code at an airport vending machine. An emotional engine checks the user's status and sends the information to a server. The server authenticates the information, and the pre-selected communication device is dispensed from the vending machine.

[1234] Examples of prompts for generative AI models

[1235] "Please explain in detail how this system's vending machines recognize user emotions and how they adjust the service content based on those emotions."

[1236] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1237] Step 1: Language Selection

[1238] Input: The user touches the vending machine's touch panel.

[1239] Specific operation: The device displays a language selection screen. The user selects their desired language using the touch panel.

[1240] Output: The terminal retrieves the language information selected by the user.

[1241] Step 2: Capture and analyze emotional states

[1242] Input: The terminal receives a signal indicating that the user has begun to operate, which is a signal that the language selection is complete.

[1243] Specific operation: The device's camera and microphone begin capturing the user's facial expressions and voice tone in real time. The emotion engine analyzes the captured data to recognize the user's emotional state (e.g., stress, satisfaction, confusion).

[1244] Output: User emotional state data is generated as a result of analysis by the emotion engine.

[1245] Step 3: Select your desired communication plan and device.

[1246] Input: The device displays an interactive interface based on the user's emotional state data and language selection information.

[1247] Specific operation: The user operates the touch panel to confirm and select their desired communication plan and communication device. The device dynamically adjusts the displayed options based on the analysis results of the emotion engine.

[1248] Output: The communication plan and device information selected by the user are registered on the device.

[1249] Step 4: Submitting information and obtaining updates

[1250] Input: The terminal generates request data that includes the user's selected communication plan and terminal information.

[1251] Specific operation: The terminal sends request data to the server. After receiving the request data, the server retrieves the latest plan and terminal information from the database and returns the calculation result to the terminal.

[1252] Output: The latest information from the server is sent to the terminal and displayed to the user.

[1253] Step 5: Payment Procedure

[1254] Input: The terminal displays the latest information retrieved from the server and the user's selection information.

[1255] Specific operation: The user enters payment information using a credit card scanner or QR code reader. The terminal collects the payment information and sends it to the server.

[1256] Output: Payment information is sent to the server and the authentication process begins.

[1257] Step 6: Authentication and device ejection

[1258] Input: The server receives the payment information and completes the authentication process.

[1259] Specific operation: The server authenticates the payment information and notifies the terminal of the result. If authentication is successful, the terminal checks the inventory information and ejects the selected communication terminal.

[1260] Output: The authenticated communication device is ejected from the terminal and received by the user.

[1261] This allows users to purchase communication devices in a simple way using emotion recognition technology, and to have a smooth purchasing experience while reducing stress and confusion through an interactive interface.

[1262] (Application Example 2)

[1263] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1264] Conventional automated vending systems often cause users to feel confused or anxious when making a purchase, resulting in a less-than-smooth selection process. Furthermore, a lack of appropriate guidance and suggestions tailored to the user's emotional state hinders the optimization of the user experience. This invention aims to solve these problems and provide a means to improve the user interface.

[1265] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1266] In this invention, the server includes means for a user to access an automated vending system for purchasing a mobile device, means for displaying an interactive interface and for the user to select a desired mobile plan and model, means for transmitting the selection to the server and providing the latest plan and model information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, means for ejecting the selected mobile device after authentication is complete, and means equipped with an emotion recognition engine that recognizes the user's emotions and dynamically adjusts the dialogue and suggestions. This makes it possible to provide appropriate guidance and suggestions according to the user's emotional state, optimizing the user experience and facilitating a smooth selection process.

[1267] An "automated vending system" is a system that allows users to easily select, purchase, and receive mobile devices and other products.

[1268] An "interactive interface" is an interface that enables two-way interaction between the user and the system, and assists in presenting options and inputting information.

[1269] An "emotion recognition engine" is a software or hardware component that analyzes and recognizes emotions from a user's facial expressions and voice tone.

[1270] "Recognizing user emotions" is the process of extracting and accurately determining a user's emotional state (e.g., stress, satisfaction, confusion, etc.) from their facial expressions and voice.

[1271] A "server" is a computer system that processes user input information and provides necessary information and services.

[1272] "Mobile phone plan" refers to the pricing plan and contract details of the mobile phone service that a user chooses.

[1273] "Model" refers to the specific model or type of mobile phone selected by the user.

[1274] "Payment information" refers to information about credit card details and other payment methods that users need to purchase products.

[1275] "Authentication" is the process of verifying and approving whether the entered information is correct.

[1276] "Dispensing" refers to the act of providing the selected product to the user from the vending machine.

[1277] This invention is a system that combines an emotion recognition engine with an automated vending system that allows users to easily select, purchase, and receive mobile devices. Detailed embodiments of this invention are described below.

[1278] overview

[1279] This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile phone plan and model. The system incorporates an emotion recognition engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection, the system confirms inventory and payment method, and upon successful authentication, the mobile phone is dispensed from the vending machine.

[1280] System Configuration

[1281] Vending machine (terminal)

[1282] The device features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion recognition engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, mobile plan and model selection, option selection, payment information entry, and a built-in QR code scanner.

[1283] server

[1284] The server processes the selections and payment information sent from the device and provides the latest plan and device information based on that information. It also authenticates the payment and inventory information and sends the results to the device. Data from the emotion recognition engine is also sent to the server for reference.

[1285] Details of the emotion recognition function

[1286] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[1287] Program implementation

[1288] Hardware and software to be used

[1289] Smartphone: A device operated by a user.

[1290] Emotion recognition software: Affectiva, Microsoft Azure Emotion API, etc.

[1291] Server: A backend system for data processing (e.g., Amazon Web Services, Firebase).

[1292] Data processing and data calculation

[1293] The server performs the following actions:

[1294] 1. Input Data Acquisition: User facial expressions and voice tone are captured in real time via the device's camera and microphone. The libraries used are WebRTC and OpenCV.

[1295] 2. Emotion Recognition: Emotion recognition software (such as Affectiva or Azure Emotion API) is used to analyze the user's emotions from the captured data. The analysis results are obtained in JSON format.

[1296] 3. Adjusting the interactive interface: Dynamically adjust the UI / UX display based on the user's emotional state. For example, if the user is frustrated, simplify the ordering process; if they are satisfied, present promotional information.

[1297] 4. Backend Integration: Selection data and sentiment data are sent to the server for storage, and appropriate information is provided in real time. The technologies used are Firebase and AWS.

[1298] As a concrete example, consider a scenario where a user opens a food delivery app on their smartphone and begins placing an order. The camera and microphone analyze the user's emotions in real time, and if it determines that the user is frustrated, a message appears asking, "Do you need help completing your order quickly?" If the user answers "yes," the process switches to completing the order in the shortest possible steps.

[1299] Example of a prompt

[1300] Develop an app that recognizes user emotions using facial expressions and voice tone, and adjusts the food delivery ordering process accordingly. For example, if a user is frustrated, the app simplifies the ordering process and helps them complete their order quickly. If the user is satisfied, it provides promotional information or suggests new menu items. Technologies used should include Affectiva, Azure Emotion API, Firebase, and AWS.

[1301] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1302] Step 1:

[1303] The user accesses the vending machine (terminal). A screen for selecting a language appears on the terminal's touch panel. Based on the user's selection, the system proceeds to the next step. The user's actions on the touch panel serve as input, and the next screen is displayed as output.

[1304] Step 2:

[1305] The device begins capturing the user's facial expressions and voice in real time using its camera and microphone. This data is then analyzed using emotion recognition software (such as Affectiva or Azure Emotion API). The input consists of the user's facial expressions and voice data, and the output is the analysis result (e.g., emotional state such as stress, satisfaction, or confusion).

[1306] Step 3:

[1307] The interactive interface is initiated, and the user sequentially selects their mobile plan and device. Based on the analysis results of the emotion recognition engine, the UI / UX display is dynamically adjusted. For example, if the system determines that the user is confused, it will either provide a more detailed explanation or simplify the options. The input consists of the analysis results of the emotional state and the user's selections, and the output is the adjusted interface.

[1308] Step 4:

[1309] Once the user's selection is confirmed, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. The input is the user's selection, and the output is the latest plan and device information sent to the device.

[1310] Step 5:

[1311] The user initiates the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The input is payment information, and the output is the payment authentication result.

[1312] Step 6:

[1313] The server authenticates the payment information and notifies the terminal of the result. If authentication is complete, the terminal checks the inventory information and ejects the selected mobile device. The inputs are payment information and inventory information, and the outputs are the authentication result and the ejection of the product.

[1314] Step 7:

[1315] Once all selection and payment processes are complete, the device sends the final data from the emotion recognition engine to the server. The server uses this data to record the user's experience and use it to improve future services. The input is the final analysis result of the emotional state, and the output is recording it in the database.

[1316] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1317] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1318] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1319] [Fourth Embodiment]

[1320] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1321] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1322] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1323] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1324] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1325] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1326] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1327] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1328] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1329] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1330] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1331] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1332] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1333] The present invention is a system that allows users to easily select, purchase, and receive a mobile device using an interactive automated vending system. Detailed embodiments of the present invention are described below.

[1334] overview

[1335] This system consists of a vending machine for purchasing mobile devices, a server that communicates with the vending machine, and a user. The user accesses the vending machine and selects the desired mobile device plan and model through an interactive interface. The system then checks the inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[1336] System Configuration

[1337] Vending machine (terminal)

[1338] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a built-in QR code scanner.

[1339] server

[1340] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[1341] Program processing

[1342] Selection process

[1343] When a user accesses the vending machine, the terminal first displays a language selection screen. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired mobile plan and device.

[1344] Information transmission and provision

[1345] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends them back to the device. This allows the user to make selections based on the latest information in real time.

[1346] Payment and Authentication

[1347] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected mobile device.

[1348] Examples

[1349] Example 1: Purchase on the spot

[1350] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and the latest smartphone model. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[1351] Example 2: Pre-order and pick-up

[1352] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at a vending machine in the airport. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine.

[1353] Thus, the system of the present invention provides a means for users to quickly and conveniently obtain a mobile device in busy airports and areas with many tourists.

[1354] The following describes the processing flow.

[1355] System processing flow - Pattern (1)

[1356] Step 1:

[1357] The user accesses the vending machine (terminal).

[1358] The device displays the initial screen and begins the operation guide.

[1359] Step 2:

[1360] The user selects the language using the touch panel.

[1361] The device displays an interactive interface according to the selected language.

[1362] Step 3:

[1363] The device displays a list of multiple mobile plans and models.

[1364] The user selects their desired plan and model using the touch panel.

[1365] Step 4:

[1366] The device sends the selected information to the server.

[1367] Based on the information received by the server, it calculates and returns the latest plan and model details.

[1368] Step 5:

[1369] The device receives the latest plan and device information from the server and displays it to the user.

[1370] The user selects options on the screen (e.g., compensation plan, accessories).

[1371] Step 6:

[1372] The terminal sends the selected option to the server.

[1373] The server calculates the final amount based on the options and sends it back.

[1374] Step 7:

[1375] The user selects a payment method.

[1376] The device displays the payment method (credit card, QR code, etc.).

[1377] Step 8:

[1378] The user enters payment information or scans a QR code.

[1379] The device sends payment information to the server.

[1380] Step 9:

[1381] The server authenticates the payment information and notifies the terminal of the authentication result.

[1382] Depending on the payment method, the system communicates with the authentication authority to confirm the payment.

[1383] Step 10:

[1384] The device receives an authentication completion notification and displays it to the user.

[1385] The terminal activates its internal mechanism and ejects the selected mobile device.

[1386] Step 11:

[1387] The user receives the mobile device.

[1388] The terminal displays the transaction completion screen, and the transaction is finished.

[1389] System processing flow - Pattern (2)

[1390] Step 1:

[1391] Users access websites via the internet.

[1392] The server displays an interactive interface, prompting the user to select a plan and model.

[1393] Step 2:

[1394] Users select their desired plan and model through an interactive interface.

[1395] The server records the selection.

[1396] Step 3:

[1397] The server displays a final confirmation screen where the user selects options and payment methods.

[1398] The user completes the payment online.

[1399] Step 4:

[1400] The server authenticates the payment information, and the authentication is completed.

[1401] The server generates a QR code containing the pre-application details and sends it to the user (via email or download).

[1402] Step 5:

[1403] The user arrives in Japan and accesses a vending machine at the airport.

[1404] The device displays a QR code scanning screen.

[1405] Step 6:

[1406] The user holds the QR code up to the vending machine's scanner.

[1407] The device reads the QR code and sends it to the server.

[1408] Step 7:

[1409] The server performs authentication based on the QR code information and sends the result to the device.

[1410] The server checks the inventory information and sends a pickup instruction to the terminal.

[1411] Step 8:

[1412] The device displays the authentication result to the user.

[1413] The terminal activates its internal mechanism and ejects a pre-selected mobile device.

[1414] Step 9:

[1415] The user receives the mobile device.

[1416] The terminal displays the transaction completion screen, and the transaction is finished.

[1417] (Example 1)

[1418] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1419] In modern society, the need for quick and easy purchase of communication devices is increasing due to busy schedules and the sudden need for communication devices while traveling. However, the existing communication device purchase process is often complicated and time-consuming. Therefore, there is a need for a way for users to quickly and efficiently select, purchase, and receive communication devices.

[1420] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1421] In this invention, the server includes means for the user to access an automated vending system for purchasing a communication terminal, means for displaying an interactive interface and for the user to select a desired communication plan and terminal, means for transmitting the selection to the server and providing the latest plan and terminal information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, and means for dispensing the selected communication terminal after authentication is complete. This enables the user to easily and quickly select and purchase a communication terminal and receive it immediately.

[1422] A "communication terminal" refers to an electronic device used for sending and receiving information.

[1423] An "automated vending system" refers to a machine or device that allows users to purchase goods on a self-service basis.

[1424] An "interactive interface" refers to a user interface that a user uses to interact with a system.

[1425] A "communication plan" refers to a pricing plan offered by a communication service provider that includes data capacity, call time, and other details.

[1426] A "server" refers to a computer system used for processing selections, providing information, authentication, and other related tasks.

[1427] A "QR code" is a type of two-dimensional barcode that enables the quick and easy reading of information.

[1428] "Payment information" refers to payment-related data, including credit card information and QR codes used for purchases.

[1429] "Inventory information" refers to data that shows the inventory status of a product.

[1430] "Authentication" refers to the procedure for verifying that the information sent is accurate.

[1431] "Dispensing" refers to the act of taking the selected product from the vending machine.

[1432] This invention provides a system that allows users to quickly and easily select, purchase, and receive a communication terminal using an interactive automated vending system. This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user who utilizes it. Specific embodiments are described below.

[1433] overview

[1434] The user accesses the vending machine and selects their desired communication plan and device through an interactive interface. The system then verifies the stock and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[1435] System Configuration

[1436] Vending machine (terminal)

[1437] The vending machine features a touch panel for user operation and includes a voice guidance function to assist with selection. The touch panel includes language selection, communication plan and device selection, option selection, payment information input, and a built-in QR code scanner.

[1438] server

[1439] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. The server also authenticates the payment and inventory information and transmits the results to the vending machine.

[1440] Selection process

[1441] When a user accesses the vending machine, a language selection screen is displayed first. After the user selects a language, an interactive interface is initiated, allowing them to sequentially select their desired communication plan and device.

[1442] Specific example:

[1443] The system starts when the user accesses a vending machine installed at the airport and touches the touch panel. The user selects Japanese, then chooses their desired data plan and latest communication device. They then press the confirmation button on the touch panel.

[1444] Information transmission and provision

[1445] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates the latest plan and device details and sends that information back to the device in real time.

[1446] Specific example:

[1447] When a user selects a 10GB data plan and the latest communication device, that information is sent from the device to the server. The server checks the inventory status, calculates the latest information, and sends it back to the device.

[1448] Payment and Authentication

[1449] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected communication terminal.

[1450] Specific example:

[1451] When a user makes a payment with a credit card, the payment information is sent from the terminal to the server. The server authenticates the card information and sends the result back to the terminal. Once authentication is complete, the terminal checks its inventory and ejects the selected communication terminal.

[1452] Example of a prompt

[1453] An example of a prompt to be input into the generating AI model is: "Follow the instructions on the touch panel, select Japanese, then choose a 10GB data plan and the latest model of communication device. After that, complete the payment with your credit card."

[1454] As described above, the system of the present invention provides a means for users to quickly and conveniently obtain communication terminals in busy airports and areas with many tourists.

[1455] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1456] Step 1:

[1457] When a user accesses the vending machine, the terminal displays a language selection screen. The user uses the touch panel to select their desired language.

[1458] Input: User language selection (touch panel input)

[1459] Output: Display of an interactive interface according to language selection.

[1460] Specific actions:

[1461] The user selects "Japanese" from the options displayed on the language selection screen and confirms their selection by tapping the "Confirm" button on the touch panel.

[1462] Step 2:

[1463] Once language selection is complete, the device will launch an interactive interface and display a screen for selecting a communication plan. The user will then select their desired communication plan using the touch panel.

[1464] Input: User selects communication plan (touch panel input)

[1465] Output: Information on the selected communication plan

[1466] Specific actions:

[1467] The user selects the "10GB Data Plan" from the multiple data plan options displayed and confirms their selection by tapping the "Next" button on the screen.

[1468] Step 3:

[1469] Next, the terminal displays a screen for selecting a communication terminal. The user selects their desired communication terminal using the touch panel.

[1470] Input: User selects communication device (touch panel input)

[1471] Output: Information about the selected communication terminal

[1472] Specific actions:

[1473] The user selects "Latest model communication device" from the multiple communication device options displayed and confirms their selection by tapping the "Next" button on the screen.

[1474] Step 4:

[1475] Once the user's selection is finalized, the device sends that information to the server. The server processes the received information and sends the updated plan and device details back to the device.

[1476] Input: User selections (communication plan and communication device)

[1477] Output: Latest plan and device details

[1478] Specific actions:

[1479] The device sends its selection information for a "10GB data plan" and a "latest model communication device" to the server. The server checks the availability and validity of the plan and sends the results back to the device.

[1480] Step 5:

[1481] The user proceeds to the payment screen and completes the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result.

[1482] Input: Payment information (credit card information or QR code scan)

[1483] Output: Authentication result

[1484] Specific actions:

[1485] The user swipes their credit card through the terminal's scanner, and the payment information is sent from the terminal to the server. The server authenticates the card information and sends the authentication result back to the terminal.

[1486] Step 6:

[1487] Once authentication is complete, the terminal checks its inventory and ejects the selected communication device.

[1488] Input: Authentication results and inventory information

[1489] Output: Discharge of the selected communication terminal

[1490] Specific actions:

[1491] After payment is authenticated, the terminal checks the inventory and dispenses the communication device selected by the user from the vending machine. The device is dispensed when the user presses the "Receive" button on the touch panel.

[1492] (Application Example 1)

[1493] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1494] Purchasing mobile devices, especially in physical stores, is a time-consuming and cumbersome process for users. Therefore, there is a need for a simple and efficient way to select, purchase, and receive mobile devices. Furthermore, it is necessary to support diverse languages ​​and payment methods, and the provision of real-time information and inventory checks are crucial.

[1495] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1496] In this invention, the server includes means for providing an application that is installed on a smartphone and interacts with vending machines in physical stores; means for issuing a QR code, authenticating the user in the vending system based on the plan and model information selected in advance, and notifying the user of the authentication result via the smartphone application; and means for providing an interactive interface that interacts with vending machines in physical stores via the smartphone application, equipped with a language selection function and a voice guidance function for user operation. This enables users to easily and efficiently select, purchase, and receive mobile devices.

[1497] An "automated vending system" is a system of machines that allows users to automatically select and purchase products (in this case, mobile devices).

[1498] An "interactive interface" is an interface that allows the user and the system to exchange information with each other while performing operations.

[1499] "Selected information" refers to information such as the user's desired mobile phone plan and model.

[1500] A "server" is a computer system that processes and manages data for the entire system and communicates with other devices.

[1501] "Latest plan and device information" refers to information on currently available mobile plans and mobile devices.

[1502] "Payment information" refers to the information necessary for payment that a user provides when purchasing a mobile device.

[1503] "Inventory information" refers to information about the remaining quantity of products (mobile devices) held by the system.

[1504] "Authentication" is the process of verifying that a user is a legitimate purchaser based on the information and payment details they provide.

[1505] "Dispensing" refers to the physical action of the vending machine handing over the mobile device selected by the user.

[1506] "Applications installed on smartphones" refer to software applications that run on mobile devices such as mobile phones.

[1507] A "vending machine installed in a physical store" refers to a vending system machine that is installed inside a physical store.

[1508] A "QR code" is a two-dimensional barcode that encodes information in a compact, readable format.

[1509] The "language selection function" is a feature that allows users to choose their operating language from multiple languages.

[1510] The "voice guidance function" is a feature that uses voice to guide users through operations and other information.

[1511] This invention is a system for users to easily select, purchase, and receive mobile devices. This objective is achieved using a smartphone application that interacts with vending machines in physical stores. The system primarily consists of the following means:

[1512] System Configuration

[1513] vending machine

[1514] The vending machines are installed in physical stores and feature touch panels for user operation. They also have built-in voice guidance and QR code scanners to assist with selection.

[1515] Smartphone applications

[1516] The smartphone application provides an interactive interface for users to select and purchase mobile devices and plans. This application includes a language selection function and supports multiple languages. Furthermore, the application interacts with a server to provide real-time updates.

[1517] Program processing details

[1518] Server Role

[1519] The server processes selections and payment information sent from the smartphone application to provide the latest plan and device information. The server also verifies payment and inventory information and performs authentication. The software used includes Swift (iOS), Kotlin (Android), Square API, and Stripe API.

[1520] Terminal role

[1521] The terminal (vending machine) receives input from the user and provides an interactive interface using a touch panel and voice guidance. It also authenticates the user using a QR code scanner and dispenses the selected mobile device.

[1522] User roles

[1523] The user launches a smartphone application, selects a language, and chooses their desired mobile device and plan. Once the selection is confirmed, the information is sent to the server and the payment process begins. After authentication is complete, the user can pick up their mobile device from a vending machine at a physical store.

[1524] Usage example

[1525] Example 1: Purchase at a shopping mall

[1526] The user launches a smartphone application at a cafe in the shopping mall and selects their language, mobile device, and plan. After completing the payment process, a QR code is issued. The user then goes to a vending machine located in the mall, scans the QR code, and the device is dispensed.

[1527] Example 2: Purchase at an electronics store in an urban area

[1528] Users select their device and plan in advance through an application from their home and complete the payment. A QR code is issued, and they proceed to a designated electronics store's vending machine. Authentication is performed by scanning the QR code, and the device is received on the spot.

[1529] Example of a prompt

[1530] The following are examples of prompts to input into the generating AI model.

[1531] Design a smartphone application for physical stores based on the following invention. The application should include language selection, mobile device plan selection, payment and authentication processes, and integration with vending machines. It should also use Swift (iOS) or Kotlin (Android), utilize the Square API and Stripe API for payment processing, and provide a natural language description of the generated program.

[1532] In this way, a system is realized that allows users to smoothly select, purchase, and receive mobile devices.

[1533] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1534] Step 1:

[1535] The user launches the smartphone application. A language selection screen appears, and the user selects their desired language. The selected language information is entered into the application, and as a result, the interactive interface begins.

[1536] Step 2:

[1537] The user selects their desired mobile device and plan. The application collects the user's selections and sends them to the server. These selections include the device model, data plan, and options. Based on this data, the server calculates the latest inventory and plan information and sends it back to the application.

[1538] Step 3:

[1539] The server processes the received selections and provides the latest plan and model information. This process uses database queries to retrieve inventory and the best pricing plan, and then sends that information to the application.

[1540] Step 4:

[1541] The user confirms their selections and proceeds to the payment screen. The application collects payment information and sends it to the server. This information may include credit card details or QR code payment details. The server processes this data and performs payment authentication. Payment authentication is performed using the Square API or Stripe API.

[1542] Step 5:

[1543] The server authenticates the payment information. First, it checks if the payment method is valid and sends the result to the application. If successful, a notification indicating that the payment has been completed is displayed to the application.

[1544] Step 6:

[1545] Once payment is confirmed, the server generates a QR code based on the authentication information and sends it to the application. The user then uses this code to retrieve their mobile device from a vending machine at a physical store.

[1546] Step 7:

[1547] The user approaches a vending machine installed in a physical store and scans a QR code generated by a smartphone application. The vending machine reads the QR code and sends the information to a server.

[1548] Step 8:

[1549] The server verifies that the QR code is valid and performs authentication. If successful, the server sends an instruction to the vending machine to eject the mobile device. The vending machine then physically ejects the device based on this instruction.

[1550] Step 9:

[1551] The user can retrieve their selected mobile device from the vending machine. After retrieval, a notification confirming receipt will appear on the smartphone application. This completes all steps in the process.

[1552] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1553] This invention is a system that combines an automated vending system that allows for easy selection, purchase, and receipt of mobile devices with an emotion engine that recognizes the user's emotions. Detailed embodiments of this invention are described below.

[1554] overview

[1555] This system consists of a vending machine (terminal) for purchasing mobile devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile plan and model, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the system checks inventory and payment method, and once authentication is complete, the mobile device is dispensed from the vending machine.

[1556] System Configuration

[1557] Vending machine (terminal)

[1558] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes language selection, mobile plan and model selection, option selection, payment information input, and a QR code scanner.

[1559] server

[1560] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and model information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[1561] Details of the emotion recognition function

[1562] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[1563] Program processing

[1564] Selection process

[1565] When a user accesses the vending machine, the terminal first displays a language selection screen, and the emotion engine begins capturing the user's facial expressions and voice tone. After the user selects a language, an interactive interface begins, allowing them to sequentially select their desired mobile plan and model. The emotion engine analyzes the user's emotional state and adjusts the guidance and options as needed.

[1566] Information transmission and provision

[1567] Once the user's selection is finalized, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. This allows the user to make selections based on the latest information in real time. Emotion engine data is also sent to the server for reference.

[1568] Payment and Authentication

[1569] After confirming their selection, the user proceeds with the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The server authenticates the payment information and notifies the terminal of the result. Once authentication is complete, the terminal checks the inventory information and ejects the selected mobile device.

[1570] Examples

[1571] Example 1: Purchase on the spot

[1572] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and voice, displaying additional explanations if they appear confused. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[1573] Example 2: Pre-order and pick-up

[1574] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at a vending machine in the airport. An emotional engine checks the user's status and sends the information to a server. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine.

[1575] Thus, by combining the system of the present invention with emotion recognition functionality, it further enhances the user experience and provides a means for obtaining a mobile device quickly and intuitively.

[1576] The following describes the processing flow.

[1577] System processing flow - an invention combining an emotion engine

[1578] Step 1:

[1579] The user accesses the vending machine (terminal).

[1580] The device displays the initial screen and begins the operation guide.

[1581] Step 2:

[1582] The user selects the language using the touch panel.

[1583] The device displays an interactive interface according to the selected language.

[1584] The device activates its emotion engine and begins capturing the user's facial expressions and voice.

[1585] Step 3:

[1586] The device displays a list of multiple mobile plans and models.

[1587] The user selects their desired plan and model using the touch panel.

[1588] The emotion engine analyzes the user's emotional state while they are making a selection.

[1589] Step 4:

[1590] If the user is confused, the device will display additional explanations or support messages.

[1591] If the user is satisfied, proceed to the next process.

[1592] Step 5:

[1593] The device sends the selected information to the server.

[1594] Based on the information received by the server, it calculates and returns the latest plan and model details.

[1595] Step 6:

[1596] The device receives the latest plan and device information from the server and displays it to the user.

[1597] The user selects options on the screen (e.g., compensation plan, accessories).

[1598] The emotion engine analyzes the user's emotional state again.

[1599] Step 7:

[1600] The terminal sends the selected option to the server.

[1601] The server calculates the final amount based on the options and sends it back.

[1602] Step 8:

[1603] The user selects a payment method.

[1604] The device displays the payment method (credit card, QR code, etc.).

[1605] Step 9:

[1606] The user enters payment information or scans a QR code.

[1607] The device sends payment information to the server.

[1608] The emotion engine monitors the user's stress level.

[1609] Step 10:

[1610] The server authenticates the payment information and notifies the terminal of the authentication result.

[1611] Depending on the payment method, the system communicates with the authentication authority to confirm the payment.

[1612] Step 11:

[1613] The device receives an authentication completion notification and displays it to the user.

[1614] The terminal activates its internal mechanism and ejects the selected mobile device.

[1615] The emotion engine performs a final check on user satisfaction.

[1616] Step 12:

[1617] The user receives the mobile device.

[1618] The terminal displays the transaction completion screen, and the transaction is finished.

[1619] The emotion engine collects user feedback and uses it to improve the system in the future.

[1620] Examples

[1621] Example 1: Purchase on the spot

[1622] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and tone of voice. If the user appears confused, the device displays detailed explanations to make the selection easier. Next, the user makes a payment with a credit card, and the server authenticates the information. Once authentication is complete, the selected smartphone is dispensed from the vending machine. The emotion engine measures the user's satisfaction and saves it as feedback.

[1623] Example 2: Pre-order and pick-up

[1624] The user selects their desired plan and device via a website from their home and completes the payment process. A QR code is then issued. Upon arrival in Japan, the user scans the QR code at an airport vending machine. An emotion engine checks the user's state and sends the information to a server. The server authenticates the information, and the pre-selected mobile device is dispensed from the vending machine. The emotion engine measures the user's satisfaction and stores it as feedback.

[1625] Thus, by combining the system of the present invention with emotion recognition functionality, it significantly improves the user experience and provides a means for obtaining a mobile device quickly and intuitively.

[1626] (Example 2)

[1627] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1628] Traditional vending systems often caused confusion and stress for users during the selection and purchase process, highlighting the need for improved user experience. Furthermore, they lacked flexible, interactive interfaces that considered user emotions, resulting in insufficient support for smooth purchase decision-making. Streamlining inventory and payment information verification and authentication processes was also required.

[1629] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for analyzing the user's facial expressions and voice and dynamically adjusting the interface and guide content based on the emotional state, means for transmitting the selection content to the server and providing the latest plan and terminal information, and means for confirming payment information and inventory information and performing authentication. This reduces the confusion and stress felt by the user and allows the purchasing process to proceed smoothly through a flexible interactive interface. In addition, the inventory confirmation and payment authentication processes are streamlined, and it becomes possible to receive the communication terminal quickly.

[1630] A "user" is someone who uses the system to purchase a communication device.

[1631] "Communication terminals" is a general term for electronic devices with communication capabilities, such as mobile phones, smartphones, and tablets.

[1632] An "automated vending system" refers to an automated vending machine installed for users to purchase communication devices.

[1633] An "interactive interface" is an interface that allows users to communicate with the system in a two-way manner and proceed with making choices and performing actions.

[1634] A "communication plan" is a contract plan that defines the terms and conditions for using data communication and call services provided by a telecommunications carrier.

[1635] An "emotion engine" is software or hardware that analyzes a user's emotional state in real time from their facial expressions and voice.

[1636] A "server" is a central control unit that communicates with terminals to perform data processing and authentication.

[1637] "Selected information" refers to the communication plan and device information chosen by the user through the interactive interface.

[1638] "Latest plan and device information" refers to the latest information on communication plans and communication devices held by the server.

[1639] "Payment information" refers to credit card information and other payment method information that users provide for purchases.

[1640] "Inventory information" refers to information regarding the quantity and types of communication terminals stored within the automated vending system.

[1641] A "QR code" is a two-dimensional barcode issued to store information that a user has selected in advance.

[1642] This invention is a system that combines an automated vending system that allows for the easy selection, purchase, and receipt of communication terminals with an emotion engine that recognizes the user's emotions. A detailed embodiment of this system is described below.

[1643] overview

[1644] This system consists of a vending machine (terminal) for purchasing communication devices, a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired communication plan and device, and the system incorporates an emotion engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection is complete, the user confirms inventory and payment method, and once authentication is complete, the communication device is dispensed from the vending machine.

[1645] System Configuration

[1646] Vending machine (terminal)

[1647] The vending machine features a touch panel for user operation and includes language selection and voice guidance functions to assist with selection. Furthermore, it incorporates an emotion engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, communication plan and device selection, option selection, payment information input, and a QR code scanner.

[1648] server

[1649] The server processes the selections and payment information transmitted from the vending machine and provides the latest plan and terminal information based on that information. It also authenticates the payment and inventory information and transmits the results to the vending machine.

[1650] Details of the emotion recognition function

[1651] The emotion engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[1652] Hardware and software to be used

[1653] 1. Hardware:

[1654] Vending machine (touch panel, camera, microphone, credit card scanner, QR code reader)

[1655] Server (equipped with a high-performance processor and large memory capacity)

[1656] 2. Software:

[1657] Programming language: Python

[1658] Emotion recognition libraries: OpenCV, Dlib

[1659] Web framework: Flask

[1660] Specific example

[1661] Example 1: Purchase on the spot

[1662] The system starts when a user accesses a vending machine installed at the airport and touches the touch panel. The user selects English, then chooses their desired data plan and the latest smartphone model. During this time, an emotion engine analyzes the user's facial expressions and voice, displaying additional explanations if they appear confused. After making a payment with a credit card, the server authenticates the information, and the selected smartphone is dispensed from the vending machine.

[1663] Example 2: Pre-order and pick-up

[1664] Users select their desired plan and device via a website from their home and complete the payment process. A QR code is then issued. Upon arrival in their destination country, the user scans the QR code at an airport vending machine. An emotional engine checks the user's status and sends the information to a server. The server authenticates the information, and the pre-selected communication device is dispensed from the vending machine.

[1665] Examples of prompts for generative AI models

[1666] "Please explain in detail how this system's vending machines recognize user emotions and how they adjust the service content based on those emotions."

[1667] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1668] Step 1: Language Selection

[1669] Input: The user touches the vending machine's touch panel.

[1670] Specific operation: The device displays a language selection screen. The user selects their desired language using the touch panel.

[1671] Output: The terminal retrieves the language information selected by the user.

[1672] Step 2: Capture and analyze emotional states

[1673] Input: The terminal receives a signal indicating that the user has begun to operate, which is a signal that the language selection is complete.

[1674] Specific operation: The device's camera and microphone begin capturing the user's facial expressions and voice tone in real time. The emotion engine analyzes the captured data to recognize the user's emotional state (e.g., stress, satisfaction, confusion).

[1675] Output: User emotional state data is generated as a result of analysis by the emotion engine.

[1676] Step 3: Select your desired communication plan and device.

[1677] Input: The device displays an interactive interface based on the user's emotional state data and language selection information.

[1678] Specific operation: The user operates the touch panel to confirm and select their desired communication plan and communication device. The device dynamically adjusts the displayed options based on the analysis results of the emotion engine.

[1679] Output: The communication plan and device information selected by the user are registered on the device.

[1680] Step 4: Submitting information and obtaining updates

[1681] Input: The terminal generates request data that includes the user's selected communication plan and terminal information.

[1682] Specific operation: The terminal sends request data to the server. After receiving the request data, the server retrieves the latest plan and terminal information from the database and returns the calculation result to the terminal.

[1683] Output: The latest information from the server is sent to the terminal and displayed to the user.

[1684] Step 5: Payment Procedure

[1685] Input: The terminal displays the latest information retrieved from the server and the user's selection information.

[1686] Specific operation: The user enters payment information using a credit card scanner or QR code reader. The terminal collects the payment information and sends it to the server.

[1687] Output: Payment information is sent to the server and the authentication process begins.

[1688] Step 6: Authentication and device ejection

[1689] Input: The server receives the payment information and completes the authentication process.

[1690] Specific operation: The server authenticates the payment information and notifies the terminal of the result. If authentication is successful, the terminal checks the inventory information and ejects the selected communication terminal.

[1691] Output: The authenticated communication device is ejected from the terminal and received by the user.

[1692] This allows users to purchase communication devices in a simple way using emotion recognition technology, and to have a smooth purchasing experience while reducing stress and confusion through an interactive interface.

[1693] (Application Example 2)

[1694] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1695] Conventional automated vending systems often cause users to feel confused or anxious when making a purchase, resulting in a less-than-smooth selection process. Furthermore, a lack of appropriate guidance and suggestions tailored to the user's emotional state hinders the optimization of the user experience. This invention aims to solve these problems and provide a means to improve the user interface.

[1696] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1697] In this invention, the server includes means for a user to access an automated vending system for purchasing a mobile device, means for displaying an interactive interface and for the user to select a desired mobile plan and model, means for transmitting the selection to the server and providing the latest plan and model information, means for collecting selected options and payment information, means for verifying payment information and inventory information and performing authentication, means for ejecting the selected mobile device after authentication is complete, and means equipped with an emotion recognition engine that recognizes the user's emotions and dynamically adjusts the dialogue and suggestions. This makes it possible to provide appropriate guidance and suggestions according to the user's emotional state, optimizing the user experience and facilitating a smooth selection process.

[1698] An "automated vending system" is a system that allows users to easily select, purchase, and receive mobile devices and other products.

[1699] An "interactive interface" is an interface that enables two-way interaction between the user and the system, and assists in presenting options and inputting information.

[1700] An "emotion recognition engine" is a software or hardware component that analyzes and recognizes emotions from a user's facial expressions and voice tone.

[1701] "Recognizing user emotions" is the process of extracting and accurately determining a user's emotional state (e.g., stress, satisfaction, confusion, etc.) from their facial expressions and voice.

[1702] A "server" is a computer system that processes user input information and provides necessary information and services.

[1703] "Mobile phone plan" refers to the pricing plan and contract details of the mobile phone service that a user chooses.

[1704] "Model" refers to the specific model or type of mobile phone selected by the user.

[1705] "Payment information" refers to information about credit card details and other payment methods that users need to purchase products.

[1706] "Authentication" is the process of verifying and approving whether the entered information is correct.

[1707] "Dispensing" refers to the act of providing the selected product to the user from the vending machine.

[1708] This invention is a system that combines an emotion recognition engine with an automated vending system that allows users to easily select, purchase, and receive mobile devices. Detailed embodiments of this invention are described below.

[1709] overview

[1710] This system consists of a vending machine (terminal), a server that communicates with the terminal, and a user. Using an interactive interface, the user selects their desired mobile phone plan and model. The system incorporates an emotion recognition engine that recognizes the user's emotions and adjusts the dialogue and suggestions accordingly. After selection, the system confirms inventory and payment method, and upon successful authentication, the mobile phone is dispensed from the vending machine.

[1711] System Configuration

[1712] Vending machine (terminal)

[1713] The device features a touch panel for user operation and includes language selection and voice guidance functions to assist with selections. Furthermore, it incorporates an emotion recognition engine that recognizes emotions from the user's facial expressions and voice tone. The touch panel includes functions for language selection, mobile plan and model selection, option selection, payment information entry, and a built-in QR code scanner.

[1714] server

[1715] The server processes the selections and payment information sent from the device and provides the latest plan and device information based on that information. It also authenticates the payment and inventory information and sends the results to the device. Data from the emotion recognition engine is also sent to the server for reference.

[1716] Details of the emotion recognition function

[1717] The emotion recognition engine analyzes the user's facial expressions and voice in real time to recognize their emotional state, such as stress, satisfaction, or confusion. Based on this recognition, it dynamically adjusts the display of the interactive interface and the content of the voice guidance. For example, if the user is confused, it can either provide a more detailed explanation or simplify the options.

[1718] Program implementation

[1719] Hardware and software to be used

[1720] Smartphone: A device operated by a user.

[1721] Emotion recognition software: Affectiva, Microsoft Azure Emotion API, etc.

[1722] Server: A backend system for data processing (e.g., Amazon Web Services, Firebase).

[1723] Data processing and data calculation

[1724] The server performs the following actions:

[1725] 1. Input Data Acquisition: User facial expressions and voice tone are captured in real time via the device's camera and microphone. The libraries used are WebRTC and OpenCV.

[1726] 2. Emotion Recognition: Emotion recognition software (such as Affectiva or Azure Emotion API) is used to analyze the user's emotions from the captured data. The analysis results are obtained in JSON format.

[1727] 3. Adjusting the interactive interface: Dynamically adjust the UI / UX display based on the user's emotional state. For example, if the user is frustrated, simplify the ordering process; if they are satisfied, present promotional information.

[1728] 4. Backend Integration: Selection data and sentiment data are sent to the server for storage, and appropriate information is provided in real time. The technologies used are Firebase and AWS.

[1729] As a concrete example, consider a scenario where a user opens a food delivery app on their smartphone and begins placing an order. The camera and microphone analyze the user's emotions in real time, and if it determines that the user is frustrated, a message appears asking, "Do you need help completing your order quickly?" If the user answers "yes," the process switches to completing the order in the shortest possible steps.

[1730] Example of a prompt

[1731] Develop an app that recognizes user emotions using facial expressions and voice tone, and adjusts the food delivery ordering process accordingly. For example, if a user is frustrated, the app simplifies the ordering process and helps them complete their order quickly. If the user is satisfied, it provides promotional information or suggests new menu items. Technologies used should include Affectiva, Azure Emotion API, Firebase, and AWS.

[1732] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1733] Step 1:

[1734] The user accesses the vending machine (terminal). A screen for selecting a language appears on the terminal's touch panel. Based on the user's selection, the system proceeds to the next step. The user's actions on the touch panel serve as input, and the next screen is displayed as output.

[1735] Step 2:

[1736] The device begins capturing the user's facial expressions and voice in real time using its camera and microphone. This data is then analyzed using emotion recognition software (such as Affectiva or Azure Emotion API). The input consists of the user's facial expressions and voice data, and the output is the analysis result (e.g., emotional state such as stress, satisfaction, or confusion).

[1737] Step 3:

[1738] The interactive interface is initiated, and the user sequentially selects their mobile plan and device. Based on the analysis results of the emotion recognition engine, the UI / UX display is dynamically adjusted. For example, if the system determines that the user is confused, it will either provide a more detailed explanation or simplify the options. The input consists of the analysis results of the emotional state and the user's selections, and the output is the adjusted interface.

[1739] Step 4:

[1740] Once the user's selection is confirmed, the device sends that information to the server. Based on the received information, the server calculates and returns the latest plan and device details. The input is the user's selection, and the output is the latest plan and device information sent to the device.

[1741] Step 5:

[1742] The user initiates the payment process. The terminal collects payment information via a credit card scanner or QR code reader and sends it to the server. The input is payment information, and the output is the payment authentication result.

[1743] Step 6:

[1744] The server authenticates the payment information and notifies the terminal of the result. If authentication is complete, the terminal checks the inventory information and ejects the selected mobile device. The inputs are payment information and inventory information, and the outputs are the authentication result and the ejection of the product.

[1745] Step 7:

[1746] Once all selection and payment processes are complete, the device sends the final data from the emotion recognition engine to the server. The server uses this data to record the user's experience and use it to improve future services. The input is the final analysis result of the emotional state, and the output is recording it in the database.

[1747] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1748] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1749] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1750] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1751] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1752] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1753] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1754] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1755] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1756] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1757] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1758] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1759] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1760] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1761] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1762] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1763] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1764] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1765] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1766] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1767] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1768] The following is further disclosed regarding the embodiments described above.

[1769] (Claim 1)

[1770] A means for users to access an automated vending system for purchasing mobile devices,

[1771] A means for displaying an interactive interface and allowing the user to select their desired mobile plan and model,

[1772] A means of sending the selection to the server and providing the latest plan and model information,

[1773] Means for collecting selected options and payment information,

[1774] A means of verifying payment and inventory information and performing authentication,

[1775] A means of ejecting the selected mobile device after authentication is complete,

[1776] A system that includes this.

[1777] (Claim 2)

[1778] The system according to claim 1, which issues a QR code and performs authentication in an automated vending system based on the plan and model information selected in advance by the user.

[1779] (Claim 3)

[1780] The system according to claim 1, which provides an interactive interface equipped with a language selection function and a voice guidance function for user operation.

[1781] "Example 1"

[1782] (Claim 1)

[1783] A means for users to access an automated vending system for purchasing communication devices,

[1784] A means of displaying an interactive interface and allowing the user to select their desired communication plan and device,

[1785] A means of sending the selection to the server and providing the latest plan and device information,

[1786] Means for collecting selected options and payment information,

[1787] A means of verifying payment and inventory information and performing authentication,

[1788] A means of ejecting the selected communication terminal after authentication is complete,

[1789] A system that includes this.

[1790] (Claim 2)

[1791] The system according to claim 1, which issues a QR code and performs authentication in an automated vending system based on the plan and terminal information selected in advance by the user.

[1792] (Claim 3)

[1793] The system according to claim 1, which provides an interactive interface equipped with a language selection function and a voice guidance function for user operation.

[1794] "Application Example 1"

[1795] (Claim 1)

[1796] A means for users to access an automated vending system for purchasing mobile devices,

[1797] A means for displaying an interactive interface and allowing the user to select their desired mobile plan and model,

[1798] A means of sending the selection to the server and providing the latest plan and model information,

[1799] Means for collecting selected options and payment information,

[1800] A means of verifying payment and inventory information and performing authentication,

[1801] A means of ejecting the selected mobile device after authentication is complete,

[1802] A means of providing an application that is installed on a smartphone and interacts with vending machines in physical stores,

[1803] A system that includes this.

[1804] (Claim 2)

[1805] The system according to claim 1, which issues a QR code, performs authentication using an automated vending system based on the plan and device information selected in advance by the user, and notifies the user of the authentication result via a smartphone application.

[1806] (Claim 3)

[1807] The system according to claim 1, which includes a language selection function and a voice guidance function for user operation, and provides an interactive interface that interacts with vending machines in physical stores via a smartphone application.

[1808] "Example 2 of combining an emotion engine"

[1809] (Claim 1)

[1810] A means for users to access an automated vending system for purchasing communication devices,

[1811] A means of displaying an interactive interface and allowing the user to select their desired communication plan and device,

[1812] A means of sending the selection to the server and providing the latest plan and device information,

[1813] Means for collecting selected options and payment information,

[1814] A means of verifying payment and inventory information and performing authentication,

[1815] A means of ejecting the selected communication terminal after authentication is complete,

[1816] A means of analyzing the user's facial expressions and voice, and dynamically adjusting the interface and guide content based on their emotional state,

[1817] A system that includes this.

[1818] (Claim 2)

[1819] The system according to claim 1, which issues a QR code and performs authentication in an automated vending system based on the plan and terminal information selected in advance by the user.

[1820] (Claim 3)

[1821] The system according to claim 1, which provides an interactive interface equipped with a language selection function and a voice guidance function for user operation.

[1822] "Application example 2 when combining with an emotional engine"

[1823] (Claim 1)

[1824] A means for users to access an automated vending system for purchasing mobile devices,

[1825] A means for displaying an interactive interface and allowing the user to select their desired mobile plan and model,

[1826] A means of sending the selection to the server and providing the latest plan and model information,

[1827] Means for collecting selected options and payment information,

[1828] A means of verifying payment and inventory information and performing authentication,

[1829] A means of ejecting the selected mobile device after authentication is complete,

[1830] A means equipped with an emotion recognition engine that recognizes the user's emotions and dynamically adjusts the content of the dialogue and suggestions,

[1831] A system that includes this.

[1832] (Claim 2)

[1833] The system according to claim 1, comprising means for issuing a QR code, authenticating the user in an automated vending system based on the plan and model information selected in advance by the user, and incorporating an emotion recognition engine that recognizes the user's emotions and dynamically adjusts the dialogue and suggested content.

[1834] (Claim 3)

[1835] The system according to claim 1, comprising a language selection function and a voice guidance function for user operation, and a means equipped with an emotion recognition engine that recognizes the user's emotions and dynamically adjusts the dialogue and suggested content. [Explanation of symbols]

[1836] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to access an automated vending system for purchasing mobile devices, A means for displaying an interactive interface and allowing the user to select their desired mobile plan and model, A means of sending the selection to the server and providing the latest plan and model information, Means for collecting selected options and payment information, A means of verifying payment and inventory information and performing authentication, A means of ejecting the selected mobile device after authentication is complete, A system that includes this.

2. The system according to claim 1, which issues a QR code and performs authentication using an automated vending system based on the plan and model information selected in advance by the user.

3. The system according to claim 1, which provides an interactive interface equipped with a language selection function and a voice guidance function for user operation.

Images