System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring current location information of a user; means for acquiring destination information and route conditions specified by the user; means for calculating an optimal route based on the acquired current location information, destination information, and route conditions and generating haptic feedback data; and means for providing the generated haptic feedback data to the user with haptic feedback.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional maps and navigation systems rely on visual information, making them inconvenient for users who have difficulty navigating safely to their destinations on their own, such as the visually impaired, elderly people unfamiliar with smartphones, and children. It is also difficult to easily find the optimal route for specific road conditions, which is why there is a demand for systems that can provide reliable route guidance. [Means for solving the problem]

[0005] As a means for solving the above problem, the present invention provides a system including a means for acquiring a user's current location information, a means for acquiring destination information and route conditions specified by the user, a means for calculating an optimal route based on the acquired current location information, destination information, and route conditions, and generating haptic feedback data, and a means for providing the generated haptic feedback data to the user along with haptic feedback.

[0006] This system can guide users through routes using tactile information without relying on vision, helping them safely reach their destination. It also provides a means to calculate a corrected route and generate new tactile feedback data if the user is heading in the wrong direction, allowing them to quickly correct their direction even if they get lost.

[0007] Furthermore, by confirming that the user has reached their destination and notifying them of the arrival via haptic feedback, the system provides a sense of security.Moreover, by providing a system that further includes a means for acquiring basic information about the user, generating and saving a unique identification number, and a means for using a generative model that learns the optimal route based on the user's past movement history and requests based on the acquired basic information, more individually optimized guidance can be realized.

[0008] "Current location information" is information for identifying the location where the user is currently located, and is mainly obtained based on GPS data or the like.

[0009] "Destination information" is information indicating the final location the user is heading for, and is data necessary for route guidance.

[0010] "Route conditions" refers to information that includes specific conditions or requests that are taken into account when calculating a route (e.g., avoiding slopes, choosing roads with a lot of foot traffic, etc.).

[0011] "Haptic feedback data" is data used to convey information to the user through tactile sensations such as vibration and pressure, and is used during navigation.

[0012] The "optimal route" is the most efficient and safe route for the user, calculated based on current location information, destination information, and route conditions.

[0013] "Haptic feedback" is a means of providing tactile information that a user can receive through a device, and is used as navigation guidance.

[0014] A "corrected route" is a route that is recalculated to guide the user in the right direction if they are going in the wrong direction.

[0015] "Basic Information" refers to information that includes the user's basic personal information (e.g., name, age, specific requests, etc.) and is used for initial system configuration and individual optimization.

[0016] "Identification Number" means a unique system-generated number that identifies a user and is associated with the user information in the database.

[0017] A "generative model" is a machine learning or AI model used to learn a user's past travel history and requests and suggest optimal routes.

[0018] "Arrival confirmation" is the process by which the system confirms that the user has reached the destination and notifies the user of this through feedback. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention provides a system for safely guiding a user to a destination using haptic feedback. This allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own. An embodiment of the present invention will be described below.

[0041] System Configuration

[0042] This system consists of a user terminal (a smartphone or dedicated device), a server connected to it, and the user. Communication between each component is carried out via a network (e.g., the Internet).

[0043] Program processing

[0044] User registration and initial settings

[0045] When a user uses the system for the first time, they start up their device, log in, and enter basic information (name, age, specific requests). The device encrypts this information and sends it to the server. The server stores the received information in a database, generates a unique user identification number, and sends it back to the device. This makes it possible to manage the user's individual data and movement history.

[0046] Route learning and configuration

[0047] When a user heads to a specific destination, they input destination information from their device. The device acquires GPS data and sends it to the server along with their current location. Based on this information, the server uses a generative AI model to calculate the optimal route. This optimal route takes into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes). The calculated route is generated as haptic feedback data and sent to the device. The device stores this haptic feedback data and uses it when necessary.

[0048] Specific examples of real-time guidance

[0049] When Elderly Person A goes to the hospital

[0050] Elderly user A holds the device and requests route guidance to the hospital. The device acquires its current location information and sends it to the server with the hospital as the destination. The server generates an optimal route based on past movement patterns (e.g., a route with few slopes) and sends this as haptic feedback data to the device. The device then uses the haptic data to guide Elderly User A using a vibration pattern. If Elderly User A goes in the wrong direction, the device will warn them with a different vibration pattern and show them the correct direction. Finally, the server confirms that Elderly User A has reached the hospital and notifies them of the completion using haptic feedback.

[0051] If your child goes to school

[0052] A child's parent pre-registers their child's route to school on the device. As the child carries the device to school, the device tracks their progress in real time and provides guidance. The server periodically receives location information and checks whether the child is progressing correctly. If the child takes a wrong turn, the server calculates a corrected route and sends new haptic feedback data. The device then provides haptic feedback accordingly, correcting the route. When the child arrives at school, a notification is sent to the parent, providing peace of mind.

[0053] The above is an embodiment of the present invention, which is a system that helps various users reach their destination safely and easily. This system provides route guidance through tactile feedback without relying on vision, so anyone can easily use it.

[0054] The processing flow will be explained below.

[0055] Program processing details

[0056] User registration and initial settings

[0057] Step 1:

[0058] The user starts up the device and enters basic information such as their name, age, email address, and password into the application's login screen.

[0059] Step 2:

[0060] The terminal encrypts the basic information entered and transmits it to the server via the network.

[0061] Step 3:

[0062] The server stores the received basic information in the database and generates a new user ID.

[0063] Step 4:

[0064] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[0065] Step 5:

[0066] The device saves the received user ID and completes the initial setup.

[0067] Route learning and configuration

[0068] Step 1:

[0069] The user operates the terminal to select a destination and start setting route guidance.

[0070] Step 2:

[0071] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[0072] Step 3:

[0073] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[0074] Step 4:

[0075] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[0076] Step 5:

[0077] The terminal stores the received haptic feedback data and provides it to the user as needed.

[0078] Real-time guidance

[0079] Step 1:

[0080] To start guidance along the route specified by the user, route guidance is started on the terminal.

[0081] Step 2:

[0082] The device periodically acquires GPS data and tracks its current location.

[0083] Step 3:

[0084] The device sends its current location information to the server.

[0085] Step 4:

[0086] The server evaluates the progress based on the current position, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[0087] Step 5:

[0088] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path and generates new haptic feedback data to send to the device.

[0089] Step 6:

[0090] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern. If the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[0091] Step 7:

[0092] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[0093] Specific examples

[0094] When Elderly Person A goes to the hospital

[0095] Step 1:

[0096] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[0097] Step 2:

[0098] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[0099] Step 3:

[0100] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[0101] Step 4:

[0102] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[0103] Step 5:

[0104] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[0105] Step 6:

[0106] When Elderly A reaches his destination, the hospital, the server confirms his arrival and sends a completion notification to the terminal, which then notifies Elderly A via haptic feedback.

[0107] In this way, the system can effectively guide the user to their destination using haptic feedback.

[0108] Example 1

[0109] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] To provide a system that enables the visually impaired, the elderly, children, etc. to safely reach their destinations on their own. In particular, to improve the safety and convenience of users by enabling them to receive route guidance without relying on visual information and by providing appropriate guidance even when heading in the wrong direction.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0112] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and travel conditions specified by the user, means for calculating an optimal route using a generative AI model based on the acquired current location information, destination information, and travel conditions, and means for generating haptic feedback data, and means for generating new haptic feedback data. This enables anyone to easily and safely reach their destination without relying on vision.

[0113] A "user terminal" is an electronic device carried by users such as visually impaired people, the elderly, and children, and is primarily used to obtain GPS data and provide tactile feedback.

[0114] The "server" is a central computer system that receives data sent from user terminals, calculates optimal routes, generates haptic feedback data, and stores the user's movement history.

[0115] "Current location information" is real-time location data of a user obtained using the GPS function of the user terminal.

[0116] "Destination information" refers to geographical information of a place that a user wants to reach, which is input into the terminal.

[0117] "Travel conditions" is information including specific requests and constraints specified by the user (e.g., avoiding slopes, width of sidewalks, etc.).

[0118] A "generative AI model" is an artificial intelligence technology used by the server, and is an algorithm that calculates the optimal route based on past data and user characteristics.

[0119] "Haptic feedback data" is data that is sent to a user terminal and is used to provide route guidance to the user in the form of vibration patterns or the like.

[0120] A "corrected route" is a new optimal route that is recalculated when the user takes a wrong turn.

[0121] "Arrival confirmation" is the process by which the server confirms that the user has reached their destination.

[0122] "Haptic feedback" is a means of physically conveying route guidance and warnings to users using the device's vibration function, etc.

[0123] System Overview

[0124] The present invention is a system for safely guiding a user to a destination using haptic feedback, which allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own.

[0125] System Configuration

[0126] The system consists of the following components:

[0127] 1. User device (e.g., smartphone or dedicated device)

[0128] 2. Server

[0129] 3. Users

[0130] Communication takes place over the Internet.

[0131] Program processing

[0132] User registration and initial settings

[0133] When a user starts up the device for the first time, they are taken to a login screen where they enter basic information such as their name, age, and any specific requests.

[0134] The terminal uses AES (Advanced Encryption Standard) to encrypt the basic information entered and sends this encrypted data to the server.

[0135] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[0136] Route learning and configuration

[0137] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[0138] The device uses a GPS module to obtain data on its current location and transmits this information to a server.

[0139] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes).

[0140] The calculated path is generated as haptic feedback data, which is sent to the device, which stores the haptic feedback data in its internal memory and uses it as needed.

[0141] Real-time guidance

[0142] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[0143] The server periodically receives the user's location information and verifies that the route is correct.

[0144] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[0145] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[0146] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data to the device as a completion notification. This feedback data conveys the end notification to the user.

[0147] Specific examples

[0148] When Elderly Person A goes to the hospital

[0149] 1. The user, elderly person A, takes the device and requests route guidance to the hospital.

[0150] 2. The device obtains its current location information using the GPS module and sends it to the server with the hospital as the destination.

[0151] 3. The server generates the optimal route based on past travel patterns and basic information.

[0152] 4. The device guides Elderly A using vibration patterns based on the haptic feedback data. For example, a right turn is indicated by three short vibrations, and a left turn is indicated by one long vibration.

[0153] 5. If Elderly Person A goes in the wrong direction, the device will warn him with five short vibrations and provide him with a recalculated correct route.

[0154] 6. The server confirms that the delivery has reached the hospital and notifies the completion with a "3-second continuous vibration."

[0155] If your child goes to school

[0156] 1. Parents register their child's route to school on the device in advance.

[0157] 2. When a child carries a device and heads to school, the device tracks their location in real time.

[0158] 3. Periodically send location information to the server to confirm correct progress.

[0159] 4. If the child takes a wrong turn, the server instantly calculates a corrective path and sends a new vibration pattern to the device.

[0160] 5. When the child reaches the destination (school), an "arrival notification" is sent to the parent, providing peace of mind to the parent.

[0161] Examples of prompt statements

[0162] "Generate route guidance for an elderly person going to the hospital based on the following information: Name - Ichiro Suzuki, Age - 70, Current location - {Current location coordinates}, Destination - {Hospital coordinates}, Past travel patterns - {Past travel history}"

[0163] This provides a system that helps users reach their destination safely and easily.

[0164] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0165] Program processing steps

[0166] User registration and initial settings

[0167] Step 1:

[0168] A user starts up the device for the first time and is directed to a login screen, where they enter basic information such as their name, age, and any specific requests.

[0169] Input: Name, age, specific requests

[0170] Output: Basic information data

[0171] Step 2:

[0172] The terminal acquires the basic information entered and encrypts it using AES (Advanced Encryption Standard).

[0173] Input: Basic information data

[0174] Output: Encrypted basic information data

[0175] Step 3:

[0176] The terminal transmits the encrypted basic information data to the server.

[0177] Input: Encrypted basic information data

[0178] Output: None (data transmission completed)

[0179] Step 4:

[0180] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[0181] Input: Encrypted basic information data

[0182] Output: A unique identification number for the user

[0183] Route learning and configuration

[0184] Step 5:

[0185] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[0186] Input: Destination information

[0187] Output: None (Destination information input complete)

[0188] Step 6:

[0189] The device uses a GPS module to obtain data on its current location and sends this information along with basic and destination information to a server.

[0190] Input: GPS data, basic information, destination information

[0191] Output: None (data transmission completed)

[0192] Step 7:

[0193] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests.

[0194] Input: GPS data, basic information, destination information, past travel history, specific requests

[0195] Output: Optimal route data

[0196] Step 8:

[0197] The server generates the calculated optimal route data as haptic feedback data and transmits this data to the terminal.

[0198] Input: Optimal route data

[0199] Output: Haptic feedback data

[0200] Step 9:

[0201] The device stores the received haptic feedback data in its internal memory and uses it as needed.

[0202] Input: Haptic feedback data

[0203] Output: None (data saving completed)

[0204] Real-time guidance

[0205] Step 10:

[0206] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[0207] Input: None (real-time location acquisition)

[0208] Output: Current location information

[0209] Step 11:

[0210] The server periodically receives the user's location information and verifies that the route is correct.

[0211] Input: Current location information

[0212] Output: Correct route confirmation result

[0213] Step 12:

[0214] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[0215] Input: haptic feedback data, current location information

[0216] Output: Vibration pattern

[0217] Step 13:

[0218] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[0219] Input: current location, inaccurate heading

[0220] Output: New haptic feedback data, vibration patterns

[0221] Step 14:

[0222] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data as a completion notification to the device, which then receives the data and notifies the user.

[0223] Input: Confirmation of arrival at destination

[0224] Output: Haptic feedback data and vibration pattern for completion notification

[0225] This allows the user to reach their destination safely and easily.

[0226] (Application example 1)

[0227] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0228] People with visual impairments, the elderly, and children face challenges in finding the products they are looking for in physical stores. Navigating safely and efficiently without relying on vision is difficult. Furthermore, there is a lack of systems that provide appropriate feedback to users when they have taken a wrong turn or when they have reached the desired product. This makes in-store shopping stressful and inconvenient for many users.

[0229] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0230] In this invention, the server includes means for acquiring user's current location information, means for acquiring user-specified target product information and in-store conditions, and means for calculating an optimal route based on the acquired current location information, target product information, and in-store conditions, and generating haptic feedback data, thereby enabling visually impaired people, the elderly, and children to move safely and efficiently within a physical store and reach their target products.

[0231] "Current location information" is data for identifying the user's current location.

[0232] "Target product information" is information about a specific product or product category that the user wants to find in the store.

[0233] "In-store conditions" refers to information about the structure and layout of the store, and conditions for route selection (e.g., passable aisles, product display positions).

[0234] "Calculate route" is a process of calculating the optimal route for the user to reach the target product based on current location information, target product information, and in-store conditions.

[0235] "Haptic feedback data" is data that a device uses to communicate information to a user through the sense of touch, specifically including sensory signals such as vibration and pressure.

[0236] A "correction route" is a new path that the user can take to get back on the correct route if they take a wrong turn.

[0237] The "arrival notification" is a notification accompanied by haptic feedback to inform the user that they have reached the desired product.

[0238] To realize this invention, a system that links the functions of a user terminal (such as a smartphone), a server, and the user is required. This system uses haptic feedback to help users find the product they are looking for in a physical store.

[0239] System Configuration

[0240] User terminal

[0241] The user terminal is a device that the user can carry around, such as a smartphone or a dedicated device. It includes location information acquisition functions such as GPS or beacons (Bluetooth) and vibration motors to provide haptic feedback. The user terminal communicates with the server via the Internet or Bluetooth.

[0242] server

[0243] The server uses cloud services such as Amazon Web Services (AWS). Platforms such as AWS EC2 and AWS Lambda are used to execute the series of processes. MySQL is also used as the database. The server receives the user's location information and desired product information, and uses a generative AI model (such as OpenAI's GPT-3) to calculate the optimal route.

[0244] Program processing

[0245] 1. User registration and initial settings

[0246] A user installs and launches the app for the first time, and enters their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[0247] This information is encrypted by the device and sent to an AWS server, which stores it in a database and generates a unique identification number for the user.

[0248] 2. In-store location and routing

[0249] When a user enters a store, the device obtains its current location using GPS or a beacon system.

[0250] Users use the app to input the desired product name or category, such as "Milk section."

[0251] The server receives this information and uses a generative AI model to calculate the optimal path and generate haptic feedback data.

[0252] 3. Real-time route guidance

[0253] Based on the haptic feedback data sent from the server, the device uses vibration patterns to provide guidance to the user.

[0254] If the user takes a wrong turn, the device calculates a corrective path and provides new haptic feedback data.

[0255] When the user arrives at the desired product, they are notified of the arrival via haptic feedback.

[0256] Specific examples

[0257] For example, if a user selects the "Milk Corner" as their destination, the app retrieves their current location and calculates the optimal route. Using a generative AI model, the app presents prompts that suggest the clearest and safest route for the user. If the user takes a wrong turn, the server calculates a new route and the device corrects the guidance with haptic feedback.

[0258] Example prompt sentence:

[0259] Japanese

[0260] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[0261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0262] Step 1:

[0263] User registration and initial settings

[0264] Users install and launch the app, then enter their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[0265] The terminal encrypts this input information and transmits it to the server.

[0266] The server stores the received information in a MySQL database, generates a unique identification number for the user, and sends it back to the terminal.

[0267] Input: User information (name, age, needs).

[0268] Output: A unique identification number for the user.

[0269] Step 2:

[0270] Obtaining location information within the store

[0271] A user enters a store.

[0272] The device obtains its current location information using GPS or a beacon system (Bluetooth).

[0273] Once the current location information is obtained, it is sent to the server.

[0274] Input: None.

[0275] Output: Current location of the user.

[0276] Step 3:

[0277] Enter the desired product information

[0278] Users enter the desired product name or category into the app, such as "Milk section."

[0279] The terminal transmits the input target product information to the server.

[0280] Input: Desired product information.

[0281] Output: None.

[0282] Step 4:

[0283] Optimal Route Calculation

[0284] Based on the received current location information and desired product information, the server calculates the optimal route using a generative AI model (e.g., OpenAI GPT-3) taking into account in-store conditions.

[0285] The server generates the calculated optimal path as haptic feedback data and transmits it to the terminal.

[0286] Input: current location information, desired product information, in-store conditions.

[0287] Output: Haptic feedback data.

[0288] Step 5:

[0289] Providing haptic feedback

[0290] The device generates a vibration pattern based on the received haptic feedback data and provides guidance to the user through haptic feedback.

[0291] Input: Haptic feedback data.

[0292] Output: Haptic feedback via vibration.

[0293] Step 6:

[0294] Check and correct heading

[0295] If the user takes a wrong turn, the device will detect the heading error based on data from beacons or GPS.

[0296] If the terminal detects an error, it sends the information to the server.

[0297] The server calculates the corrective path and generates new haptic feedback data to send to the terminal.

[0298] Input: Heading error information.

[0299] Output: New haptic feedback data.

[0300] Step 7:

[0301] Confirmation that the desired product has arrived

[0302] When the user reaches the desired product, the terminal confirms arrival based on the location information.

[0303] The server generates data to notify the device of the arrival using haptic feedback and sends it to the device.

[0304] Input: Arrival location information.

[0305] Output: Reach haptic feedback data.

[0306] For example, here is the prompt again:

[0307] Japanese

[0308] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[0309] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0310] The present invention is a system that recognizes a user's emotions and provides adaptive haptic feedback. This system is composed of a user terminal, a server, and a user, and has an emotion engine built in.

[0311] System Configuration

[0312] The user terminal (smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[0313] User registration and initial settings

[0314] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[0315] Route learning and configuration

[0316] The user inputs their destination, and the device sends their current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past travel history, and specific requests. Haptic feedback data is generated and sent to the device. The device stores this data and provides haptic feedback when needed.

[0317] Emotion Recognition and Adaptive Feedback

[0318] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or tense, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[0319] Specific examples

[0320] When Elderly Person A goes to the hospital

[0321] Elderly user A starts the device and requests route guidance to the hospital. The device acquires current location information and sends destination information to the server. The server generates an optimal route and sends haptic feedback data to the device. The device provides appropriate haptic feedback to Elderly User A. If Elderly User A goes in the wrong direction, the device provides warning haptic feedback and suggests the correct direction.

[0322] If the device detects that Elderly A is feeling stressed or tense during the interaction, the device's emotion engine sends that information to the server, which can then adjust the feedback pattern and resend the haptic feedback to provide a sense of security.

[0323] If your child goes to school

[0324] A parent sets their child's school route on the device. As the child heads to school, the device tracks their progress in real time and provides route guidance. The server periodically receives location information to check whether the user is heading in the correct direction. If the user heads in the wrong direction, the server calculates a corrected route and sends new haptic feedback data to the device, which then provides a warning.

[0325] If the child feels tired during the process, the emotion engine sends that information to the server, which then provides haptic feedback to encourage them to take a break.

[0326] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[0327] The processing flow will be explained below.

[0328] Program processing details

[0329] User registration and initial settings

[0330] Step 1:

[0331] The user starts up the device and enters basic information such as their name, age, email address, and password on the login screen.

[0332] Step 2:

[0333] The terminal encrypts the basic information entered and transmits it to the server via the network.

[0334] Step 3:

[0335] The server stores the received basic information in the database and generates a new user ID.

[0336] Step 4:

[0337] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[0338] Step 5:

[0339] The device stores the received user ID and initializes the emotion engine to collect initial data to generate the user's emotion profile.

[0340] Route learning and configuration

[0341] Step 1:

[0342] The user operates the terminal to select a destination and start setting route guidance.

[0343] Step 2:

[0344] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[0345] Step 3:

[0346] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[0347] Step 4:

[0348] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[0349] Step 5:

[0350] The terminal stores the received haptic feedback data and provides it to the user as needed.

[0351] Emotion Recognition and Adaptive Feedback

[0352] Step 1:

[0353] Guidance will begin along the route specified by the user on the terminal.

[0354] Step 2:

[0355] The device periodically acquires GPS data and tracks its current location.

[0356] Step 3:

[0357] The device recognizes the user's emotions using sensor data such as voice, facial expressions, and heart rate.

[0358] Step 4:

[0359] The device transmits the recognized emotion data to the server.

[0360] Step 5:

[0361] The server evaluates the progress and emotion data based on the current location, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[0362] Step 6:

[0363] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path, generates new haptic feedback data, and sends it to the terminal.

[0364] Step 7:

[0365] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern: if the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[0366] Step 8:

[0367] The device adjusts the feedback pattern according to the user's emotions, providing haptic feedback to relieve stress and tension, while simultaneously providing haptic feedback to encourage rest if the user's emotions indicate fatigue.

[0368] Step 9:

[0369] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[0370] Specific examples

[0371] When Elderly Person A goes to the hospital

[0372] Step 1:

[0373] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[0374] Step 2:

[0375] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[0376] Step 3:

[0377] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[0378] Step 4:

[0379] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[0380] Step 5:

[0381] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[0382] Step 6:

[0383] The device recognizes the emotions of elderly person A using voice, facial expression, and heart rate sensors, and transmits the emotional data to the server.

[0384] Step 7:

[0385] The server adjusts the feedback pattern based on the received emotional data, regenerates haptic feedback that provides a sense of security, and sends it to the device.

[0386] Step 8:

[0387] When Elderly A reaches the hospital, the server confirms the arrival and sends a completion notification to the terminal, which then notifies Elderly A of the arrival via haptic feedback.

[0388] If your child goes to school

[0389] Step 1:

[0390] Parents can register their children's routes to school in advance on their devices.

[0391] Step 2:

[0392] As a child walks to school, the device tracks their progress in real time and provides route guidance.

[0393] Step 3:

[0394] The server periodically receives location information and checks whether the user is heading in the correct direction.

[0395] Step 4:

[0396] If a wrong turn occurs, the server calculates a corrective path and sends new haptic feedback data to the device.

[0397] Step 5:

[0398] The device will guide you in the right direction based on the new haptic feedback it receives, and adjust the vibration pattern to a more reassuring one if the user is feeling stressed or tense.

[0399] Step 6:

[0400] If the device senses fatigue, it will provide haptic feedback to encourage you to take a break.

[0401] Step 7:

[0402] When the child arrives at school, the server confirms the arrival, sends a notification of arrival to the terminal, and notifies the parents.

[0403] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[0404] Example 2

[0405] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0406] Conventional route guidance systems can obtain a user's current location information and provide the optimal route to a destination, but they do not provide feedback that takes into account the user's emotional state. In particular, there is a lack of means to help users with special needs, such as the elderly and children, safely reach their destination while reducing stress and tension. Furthermore, there is a need for a system that is flexible enough to provide appropriate real-time feedback regarding incorrect directions.

[0407] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and requirements specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and requirements and generating haptic feedback data using a generative AI model, means for providing the generated haptic feedback data to the user together with haptic feedback, means for collecting sensor data and recognizing the user's emotions in real time, and means for adjusting the haptic feedback pattern according to the user's emotions. This not only enables the user to reach their destination safely, but also makes it possible to provide appropriate feedback in real time according to errors in heading or the user's emotional state.

[0408] "Means for obtaining the user's current location information" refers to a function for identifying and obtaining the user's current geographical location using a GPS module or other location information services from the device the user is using.

[0409] The "means for obtaining user-specified destination information and requirements" refers to a function for collecting the specific destination and the user's specific wishes and requests that the user has input into the system.

[0410] "Means for calculating the optimal route and generating haptic feedback data using a generative AI model" refers to a function that uses an AI model to calculate the optimal route for the user based on the acquired current location information, destination information, and requirements, and creates data for haptic feedback.

[0411] The "means for providing the generated haptic feedback data to the user along with haptic feedback" is a function for using the generated haptic feedback data to provide a haptic notification to the user's device.

[0412] "Means for collecting sensor data and recognizing user emotions in real time" refers to a function that uses sensors installed in the user's device to collect and analyze data such as voice, facial expressions, and heart rate, in order to recognize the user's emotional state in real time.

[0413] The "means for adjusting the haptic feedback pattern in accordance with the user's emotion" is a function for appropriately changing the intensity and pattern of the haptic feedback based on the recognized emotion of the user.

[0414] MODE FOR CARRYING OUT THE INVENTION

[0415] The present invention provides a system for recognizing a user's emotions and adaptively providing haptic feedback. The system is comprised of a user terminal, a server, and a user, and has an emotion engine built in. Specific embodiments of the system are described below.

[0416] System Configuration

[0417] The user terminal (e.g., a smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[0418] Specific functions and operations

[0419] User registration and initial settings

[0420] 1. The user starts the device and enters basic information (name, age, specific requests).

[0421] 2. The device encrypts the entered information and sends it to the server.

[0422] 3. The server stores the information in a database, generates a user ID, and sends it to the device.

[0423] 4. The device collects initial data such as voice samples and heart rate to create an emotional profile.

[0424] Route learning and configuration

[0425] 1. The user enters the destination, and the device obtains the current location information and sends it to the server.

[0426] 2. The server uses a generative AI model to calculate the optimal route based on the current location, destination, past travel history, and specific requests.

[0427] 3. The server generates haptic feedback data and sends it to the device.

[0428] 4. The device stores the received data and provides haptic feedback when necessary.

[0429] Emotion Recognition and Adaptive Feedback

[0430] 1. The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate.

[0431] 2. The device analyzes the data collected and recognizes the user's emotions.

[0432] 3. The device sends the recognition results to the server.

[0433] 4. The server adjusts the haptic feedback pattern according to the user's emotional state and sends it back to the device.

[0434] 5. The device applies the new feedback pattern and provides haptic feedback to the user.

[0435] Specific examples

[0436] Example 1: Elderly person A goes to the hospital

[0437] 1. The user (elderly person A) starts up the device and requests "route guidance to the hospital."

[0438] 2. The device obtains the current location information using the GPS module and sends the destination information (hospital) to the server.

[0439] 3. The server generates the optimal path and sends haptic feedback data to the device.

[0440] 4. The device displays route directions and provides haptic feedback.

[0441] 5. If Elderly Person A goes in the wrong direction, the device will generate haptic feedback to warn him and show him the correct direction to go.

[0442] 6. If Elderly Person A feels stressed during the exercise, the device's emotion engine detects an increase in heart rate and sends the emotion data to the server.

[0443] 7. The server calculates reassuring haptic feedback and sends it to the device.

[0444] 8. The device provides new haptic feedback to reassure Elderly A.

[0445] Example 2: When a child goes to school

[0446] 1. Parents set their child's school route on the device.

[0447] 2. When the child starts going to school, the device tracks their current location and sends it to the server.

[0448] 3. The server periodically checks the location information and monitors the direction of travel.

[0449] 4. If the child takes a wrong turn, the server calculates a corrective path and sends new haptic feedback data to the device.

[0450] 5. The device will provide warnings and give you precise directions to follow.

[0451] 6. The device's emotion engine detects the child's fatigue and sends the data to the server.

[0452] 7. The server provides haptic feedback to encourage a break, and the device notifies the child.

[0453] Prompt Sentence Examples

[0454] Route guidance request

[0455] The user types into the terminal, "I want to go to the hospital. Please tell me the best route."

[0456] Emotional state report

[0457] The user types into the device, "I'm feeling a little stressed right now. Please provide some haptic feedback to help me relax."

[0458] As described above, this system recognizes the user's emotions in real time and provides optimal haptic feedback, allowing the user to reach their destination with peace of mind. Furthermore, by providing appropriate feedback in real time depending on the user's emotional state and direction, the system offers a safer and more comfortable travel experience.

[0459] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0460] Step 1: User registration and initial setup

[0461] 1. Start the device and enter basic information

[0462] The user turns on the device and enters basic information such as name, age and specific requests.

[0463] Input: Basic information (name, age, specific requests)

[0464] The terminal encrypts the information entered.

[0465] Output: Encrypted basic information

[0466] 2. Transmission and storage of information

[0467] The device sends the encrypted information to the server.

[0468] Input: Encrypted basic information

[0469] The server stores the received information in a database and generates a user ID.

[0470] Output: User ID

[0471] The server sends the generated user ID to the terminal.

[0472] 3. Initial Data Collection

[0473] The device collects initial data such as voice samples and heart rate to create an emotional profile of the user.

[0474] Input: User voice samples, heart rate, etc.

[0475] The device sends the collected data to a server to create an emotional profile.

[0476] Output: Emotion profile

[0477] Step 2: Route learning and configuration

[0478] 1. Set your destination

[0479] The user inputs a destination into the terminal.

[0480] Input: Destination information

[0481] The device acquires the current location information and sends it to the server.

[0482] Input: Current location information

[0483] Output: Current location information, destination information

[0484] 2. Route calculation

[0485] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past travel history, and specific requests.

[0486] Input: current location, destination, past travel history, specific requests

[0487] Output: Optimal route data

[0488] The server generates haptic feedback data based on the calculation results.

[0489] Output: Haptic feedback data

[0490] 3. Sending and storing feedback data

[0491] The server transmits the generated haptic feedback data to the terminal.

[0492] Input: Haptic feedback data

[0493] The device stores the received data and provides haptic feedback if necessary.

[0494] Output: Stored haptic feedback data

[0495] Step 3: Emotion recognition and adaptive feedback

[0496] 1. Sensor data collection

[0497] The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate in real time.

[0498] Input: Voice data, facial expression data, heart rate data

[0499] Output: Collected sensor data

[0500] 2. Emotional Recognition

[0501] The device's emotion engine analyzes the collected data and recognizes the user's emotions.

[0502] Input: Collected sensor data

[0503] Output: Emotional state data

[0504] The device sends the analysis results to the server.

[0505] 3. Adjusting feedback patterns

[0506] The server optimizes the haptic feedback pattern based on the user's emotional state.

[0507] Input: Emotional state data

[0508] Output: Tuned feedback pattern

[0509] The server retransmits the adjusted feedback pattern to the terminal.

[0510] The device applies the new feedback pattern and provides haptic feedback to the user.

[0511] (Application example 2)

[0512] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0513] In modern shopping experiences, users are often overwhelmed by a large amount of information, which can lead to feelings of stress and anxiety. However, there is a lack of technology that can recognize these emotions in real time and respond appropriately. In particular, virtual stores are required to provide appropriate feedback based on the user's emotional state, but no effective system exists for this purpose.

[0514] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0515] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and route conditions specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and route conditions and generating haptic feedback data, means for recognizing the user's emotions in real time and transmitting the data to the server, and means for receiving an optimal haptic feedback pattern from the server and providing it to the user. This makes it possible to reduce the stress and anxiety felt by the user during their shopping experience in a virtual store and provide appropriate haptic feedback and audio feedback.

[0516] "Current location information" is data indicating the location where the user is currently located.

[0517] "Destination information" is data indicating the location the user is trying to reach.

[0518] "Route conditions" are data indicating restrictions and requests regarding the route specified by the user.

[0519] "Haptic feedback data" is data that provides a physical response to the user's senses.

[0520] "Means for real-time emotion recognition" refers to a technical mechanism for instantly determining a user's current emotional state.

[0521] A "server" is a computer system that processes and stores data over a network.

[0522] "Feedback based on emotional state" refers to tactile, audio, or other responses provided in response to the user's emotions.

[0523] A "virtual store" is a virtual shopping environment created on the Internet.

[0524] "Haptic feedback" is a technology that conveys information to a user through physical sensation.

[0525] The present invention provides a system that recognizes a user's emotions in real time and provides adaptive haptic feedback. This system is comprised of a user terminal, a server, and a user, and has an emotion engine built in.

[0526] System Configuration

[0527] The user terminal (smart glasses or smartphone) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time to generate optimal haptic feedback.

[0528] User registration and initial settings

[0529] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[0530] Route learning and configuration

[0531] When a user starts shopping in a virtual store, the device sends its current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past behavior history, and specific requests. It generates haptic feedback data and sends it to the device. The device stores this data and provides haptic feedback when needed.

[0532] Emotion Recognition and Adaptive Feedback

[0533] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or anxious, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[0534] Specific examples

[0535] A user wears smart glasses and visits a virtual store. While searching for a product in the store, the emotion engine detects the user's anxiety and sends the emotion data to the server. The server calculates the optimal haptic feedback pattern and sends it to the device. The device then provides vibration and audio feedback, reducing the user's stress.

[0536] Prompt Sentence Examples

[0537] It recognizes the user's emotions while selecting products in a virtual store and provides immediate relaxation feedback.

[0538] It detects the stress felt by users searching for a specific product in a virtual store and provides haptic feedback.

[0539] This system and method provide optimal haptic feedback based on the user's emotional state, enabling a comfortable shopping experience in a virtual store.

[0540] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0541] Step 1:

[0542] The user turns on the device and enters basic information (name, age, specific requests).

[0543] Specific operation: The user enters their name, age, and specific requests into the terminal, and then sends that information to the server by pressing the send button.

[0544] Input: User's basic information (name, age, specific needs).

[0545] Data processing / calculation: The basic information entered by the terminal is encrypted and sent to the server.

[0546] Output: Encrypted user data.

[0547] Step 2:

[0548] The server stores the received user information in a database, generates a user ID, and sends it to the terminal.

[0549] Specific operation: The server saves the received user information in a database, generates a new user ID, and returns the ID to the device.

[0550] Input: Encrypted user data.

[0551] Data processing / calculation: Saving user data and generating user IDs.

[0552] Output: The generated user ID.

[0553] Step 3:

[0554] The device collects initial data to generate an emotional profile of the user.

[0555] Specific operation: The device records initial emotion data using the camera, microphone, heart rate sensor, etc.

[0556] Input: User's voice, facial expressions, heart rate, etc.

[0557] Data processing / computation: Collecting and recording emotional data.

[0558] Output: Initial emotion profile data.

[0559] Step 4:

[0560] When a user starts shopping in a virtual store, the terminal transmits current location information to the server.

[0561] Specific operation: The device uses the GPS module to obtain its current location information and sends it to the server.

[0562] Input: The user's current location.

[0563] Data processing / calculation: Acquisition and transmission of location information.

[0564] Output: Current location information sent to the server.

[0565] Step 5:

[0566] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past behavior, and specific requests.

[0567] Specific operation: Based on the data received by the server, the generative AI model is used to calculate the optimal shopping route.

[0568] Input: current location, destination, past activity history, specific requests.

[0569] Data processing / calculation: Route calculation using generative AI models.

[0570] Output: Optimal route data.

[0571] Step 6:

[0572] Haptic feedback data is generated based on the optimal path and transmitted to the device.

[0573] Specific operation: The server generates haptic feedback data based on the route data calculated and sends it to the device.

[0574] Input: Optimal route data.

[0575] Data processing / computation: Generation of haptic feedback data.

[0576] Output: Haptic feedback data.

[0577] Step 7:

[0578] The terminal stores the received haptic feedback data and provides haptic feedback when necessary.

[0579] Specific operation: The device stores the received data in its internal storage and provides it to the user via the haptic feedback device when necessary.

[0580] Input: Haptic feedback data.

[0581] Data processing / calculation: Storing data and giving instructions to the haptic device.

[0582] Output: Haptic feedback.

[0583] Step 8:

[0584] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate.

[0585] Specific operation: The device inputs data acquired by sensors into the emotion engine and analyzes the user's emotions in real time.

[0586] Input: speech, facial expression, and heart rate data.

[0587] Data processing / computation: Analysis of emotion data.

[0588] Output: User's emotional state data.

[0589] Step 9:

[0590] If the user is feeling stressed or anxious, the device sends that information to the server.

[0591] Specific operation: The terminal transmits the user's emotional state to the server, and the server receives it.

[0592] Input: Emotional state data.

[0593] Data processing / calculation: Sending and receiving emotional data.

[0594] Output: Emotion data sent to the server.

[0595] Step 10:

[0596] The server calculates the optimal haptic feedback pattern and sends it to the device.

[0597] Specific operation: Based on the emotional state, the server adjusts the haptic feedback, generates new patterns, and sends them to the device.

[0598] Input: Emotional state data.

[0599] Data processing / calculation: Generation of optimal haptic feedback patterns.

[0600] Output: New haptic feedback data.

[0601] Step 11:

[0602] The device provides new haptic feedback to the user, reducing stress and anxiety.

[0603] Specific operation: The device activates the haptic feedback device based on the new haptic feedback data, providing the user with a sense of security.

[0604] Input: New haptic feedback data.

[0605] Data processing / computation: Implementing haptic feedback.

[0606] Output: Improved user emotional state.

[0607] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0608] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0609] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0610] [Second embodiment]

[0611] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0612] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0613] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0614] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0615] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0616] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0617] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0618] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0619] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0620] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0621] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0622] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0623] The present invention provides a system for safely guiding a user to a destination using haptic feedback. This allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own. An embodiment of the present invention will be described below.

[0624] System Configuration

[0625] This system consists of a user terminal (a smartphone or dedicated device), a server connected to it, and the user. Communication between each component is carried out via a network (e.g., the Internet).

[0626] Program processing

[0627] User registration and initial settings

[0628] When a user uses the system for the first time, they start up their device, log in, and enter basic information (name, age, specific requests). The device encrypts this information and sends it to the server. The server stores the received information in a database, generates a unique user identification number, and sends it back to the device. This makes it possible to manage the user's individual data and movement history.

[0629] Route learning and configuration

[0630] When a user heads to a specific destination, they input destination information from their device. The device acquires GPS data and sends it to the server along with their current location. Based on this information, the server uses a generative AI model to calculate the optimal route. This optimal route takes into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes). The calculated route is generated as haptic feedback data and sent to the device. The device stores this haptic feedback data and uses it when necessary.

[0631] Specific examples of real-time guidance

[0632] When Elderly Person A goes to the hospital

[0633] Elderly user A holds the device and requests route guidance to the hospital. The device acquires its current location information and sends it to the server with the hospital as the destination. The server generates an optimal route based on past movement patterns (e.g., a route with few slopes) and sends this as haptic feedback data to the device. The device then uses the haptic data to guide Elderly User A using a vibration pattern. If Elderly User A goes in the wrong direction, the device will warn them with a different vibration pattern and show them the correct direction. Finally, the server confirms that Elderly User A has reached the hospital and notifies them of the completion using haptic feedback.

[0634] If your child goes to school

[0635] A child's parent pre-registers their child's route to school on the device. As the child carries the device to school, the device tracks their progress in real time and provides guidance. The server periodically receives location information and checks whether the child is progressing correctly. If the child takes a wrong turn, the server calculates a corrected route and sends new haptic feedback data. The device then provides haptic feedback accordingly, correcting the route. When the child arrives at school, a notification is sent to the parent, providing peace of mind.

[0636] The above is an embodiment of the present invention, which is a system that helps various users reach their destination safely and easily. This system provides route guidance through tactile feedback without relying on vision, so anyone can easily use it.

[0637] The processing flow will be explained below.

[0638] Program processing details

[0639] User registration and initial settings

[0640] Step 1:

[0641] The user starts up the device and enters basic information such as their name, age, email address, and password into the application's login screen.

[0642] Step 2:

[0643] The terminal encrypts the basic information entered and transmits it to the server via the network.

[0644] Step 3:

[0645] The server stores the received basic information in the database and generates a new user ID.

[0646] Step 4:

[0647] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[0648] Step 5:

[0649] The device saves the received user ID and completes the initial setup.

[0650] Route learning and configuration

[0651] Step 1:

[0652] The user operates the terminal to select a destination and start setting route guidance.

[0653] Step 2:

[0654] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[0655] Step 3:

[0656] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[0657] Step 4:

[0658] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[0659] Step 5:

[0660] The terminal stores the received haptic feedback data and provides it to the user as needed.

[0661] Real-time guidance

[0662] Step 1:

[0663] To start guidance along the route specified by the user, route guidance is started on the terminal.

[0664] Step 2:

[0665] The device periodically acquires GPS data and tracks its current location.

[0666] Step 3:

[0667] The device sends its current location information to the server.

[0668] Step 4:

[0669] The server evaluates the progress based on the current position, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[0670] Step 5:

[0671] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path and generates new haptic feedback data to send to the device.

[0672] Step 6:

[0673] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern. If the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[0674] Step 7:

[0675] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[0676] Specific examples

[0677] When Elderly Person A goes to the hospital

[0678] Step 1:

[0679] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[0680] Step 2:

[0681] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[0682] Step 3:

[0683] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[0684] Step 4:

[0685] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[0686] Step 5:

[0687] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[0688] Step 6:

[0689] When Elderly A reaches his destination, the hospital, the server confirms his arrival and sends a completion notification to the terminal, which then notifies Elderly A via haptic feedback.

[0690] In this way, the system can effectively guide the user to their destination using haptic feedback.

[0691] Example 1

[0692] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0693] To provide a system that enables the visually impaired, the elderly, children, etc. to safely reach their destinations on their own. In particular, to improve the safety and convenience of users by enabling them to receive route guidance without relying on visual information and by providing appropriate guidance even when heading in the wrong direction.

[0694] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0695] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and travel conditions specified by the user, means for calculating an optimal route using a generative AI model based on the acquired current location information, destination information, and travel conditions, and means for generating haptic feedback data, and means for generating new haptic feedback data. This enables anyone to easily and safely reach their destination without relying on vision.

[0696] A "user terminal" is an electronic device carried by users such as visually impaired people, the elderly, and children, and is primarily used to obtain GPS data and provide tactile feedback.

[0697] The "server" is a central computer system that receives data sent from user terminals, calculates optimal routes, generates haptic feedback data, and stores the user's movement history.

[0698] "Current location information" is real-time location data of a user obtained using the GPS function of the user terminal.

[0699] "Destination information" refers to geographical information of a place that a user wants to reach, which is input into the terminal.

[0700] "Travel conditions" is information including specific requests and constraints specified by the user (e.g., avoiding slopes, width of sidewalks, etc.).

[0701] A "generative AI model" is an artificial intelligence technology used by the server, and is an algorithm that calculates the optimal route based on past data and user characteristics.

[0702] "Haptic feedback data" is data that is sent to a user terminal and is used to provide route guidance to the user in the form of vibration patterns or the like.

[0703] A "corrected route" is a new optimal route that is recalculated when the user takes a wrong turn.

[0704] "Arrival confirmation" is the process by which the server confirms that the user has reached their destination.

[0705] "Haptic feedback" is a means of physically conveying route guidance and warnings to users using the device's vibration function, etc.

[0706] System Overview

[0707] The present invention is a system for safely guiding a user to a destination using haptic feedback, which allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own.

[0708] System Configuration

[0709] The system consists of the following components:

[0710] 1. User device (e.g., smartphone or dedicated device)

[0711] 2. Server

[0712] 3. Users

[0713] Communication takes place over the Internet.

[0714] Program processing

[0715] User registration and initial settings

[0716] When a user starts up the device for the first time, they are taken to a login screen where they enter basic information such as their name, age, and any specific requests.

[0717] The terminal uses AES (Advanced Encryption Standard) to encrypt the basic information entered and sends this encrypted data to the server.

[0718] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[0719] Route learning and configuration

[0720] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[0721] The device uses a GPS module to obtain data on its current location and transmits this information to a server.

[0722] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes).

[0723] The calculated path is generated as haptic feedback data, which is sent to the device, which stores the haptic feedback data in its internal memory and uses it as needed.

[0724] Real-time guidance

[0725] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[0726] The server periodically receives the user's location information and verifies that the route is correct.

[0727] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[0728] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[0729] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data to the device as a completion notification. This feedback data conveys the end notification to the user.

[0730] Specific examples

[0731] When Elderly Person A goes to the hospital

[0732] 1. The user, elderly person A, takes the device and requests route guidance to the hospital.

[0733] 2. The device obtains its current location information using the GPS module and sends it to the server with the hospital as the destination.

[0734] 3. The server generates the optimal route based on past travel patterns and basic information.

[0735] 4. The device guides Elderly A using vibration patterns based on the haptic feedback data. For example, a right turn is indicated by three short vibrations, and a left turn is indicated by one long vibration.

[0736] 5. If Elderly Person A goes in the wrong direction, the device will warn him with five short vibrations and provide him with a recalculated correct route.

[0737] 6. The server confirms that the delivery has reached the hospital and notifies the completion with a "3-second continuous vibration."

[0738] If your child goes to school

[0739] 1. Parents register their child's route to school on the device in advance.

[0740] 2. When a child carries a device and heads to school, the device tracks their location in real time.

[0741] 3. Periodically send location information to the server to confirm correct progress.

[0742] 4. If the child takes a wrong turn, the server instantly calculates a corrective path and sends a new vibration pattern to the device.

[0743] 5. When the child reaches the destination (school), an "arrival notification" is sent to the parent, providing peace of mind to the parent.

[0744] Examples of prompt statements

[0745] "Generate route guidance for an elderly person going to the hospital based on the following information: Name - Ichiro Suzuki, Age - 70, Current location - {Current location coordinates}, Destination - {Hospital coordinates}, Past travel patterns - {Past travel history}"

[0746] This provides a system that helps users reach their destination safely and easily.

[0747] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0748] Program processing steps

[0749] User registration and initial settings

[0750] Step 1:

[0751] A user starts up the device for the first time and is directed to a login screen, where they enter basic information such as their name, age, and any specific requests.

[0752] Input: Name, age, specific requests

[0753] Output: Basic information data

[0754] Step 2:

[0755] The terminal acquires the basic information entered and encrypts it using AES (Advanced Encryption Standard).

[0756] Input: Basic information data

[0757] Output: Encrypted basic information data

[0758] Step 3:

[0759] The terminal transmits the encrypted basic information data to the server.

[0760] Input: Encrypted basic information data

[0761] Output: None (data transmission completed)

[0762] Step 4:

[0763] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[0764] Input: Encrypted basic information data

[0765] Output: A unique identification number for the user

[0766] Route learning and configuration

[0767] Step 5:

[0768] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[0769] Input: Destination information

[0770] Output: None (Destination information input complete)

[0771] Step 6:

[0772] The device uses a GPS module to obtain data on its current location and sends this information along with basic and destination information to a server.

[0773] Input: GPS data, basic information, destination information

[0774] Output: None (data transmission completed)

[0775] Step 7:

[0776] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests.

[0777] Input: GPS data, basic information, destination information, past travel history, specific requests

[0778] Output: Optimal route data

[0779] Step 8:

[0780] The server generates the calculated optimal route data as haptic feedback data and transmits this data to the terminal.

[0781] Input: Optimal route data

[0782] Output: Haptic feedback data

[0783] Step 9:

[0784] The device stores the received haptic feedback data in its internal memory and uses it as needed.

[0785] Input: Haptic feedback data

[0786] Output: None (data saving completed)

[0787] Real-time guidance

[0788] Step 10:

[0789] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[0790] Input: None (real-time location acquisition)

[0791] Output: Current location information

[0792] Step 11:

[0793] The server periodically receives the user's location information and verifies that the route is correct.

[0794] Input: Current location information

[0795] Output: Correct route confirmation result

[0796] Step 12:

[0797] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[0798] Input: haptic feedback data, current location information

[0799] Output: Vibration pattern

[0800] Step 13:

[0801] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[0802] Input: current location, inaccurate heading

[0803] Output: New haptic feedback data, vibration patterns

[0804] Step 14:

[0805] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data as a completion notification to the device, which then receives the data and notifies the user.

[0806] Input: Confirmation of arrival at destination

[0807] Output: Haptic feedback data and vibration pattern for completion notification

[0808] This allows the user to reach their destination safely and easily.

[0809] (Application example 1)

[0810] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0811] People with visual impairments, the elderly, and children face challenges in finding the products they are looking for in physical stores. Navigating safely and efficiently without relying on vision is difficult. Furthermore, there is a lack of systems that provide appropriate feedback to users when they have taken a wrong turn or when they have reached the desired product. This makes in-store shopping stressful and inconvenient for many users.

[0812] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0813] In this invention, the server includes means for acquiring user's current location information, means for acquiring user-specified target product information and in-store conditions, and means for calculating an optimal route based on the acquired current location information, target product information, and in-store conditions, and generating haptic feedback data, thereby enabling visually impaired people, the elderly, and children to move safely and efficiently within a physical store and reach their target products.

[0814] "Current location information" is data for identifying the user's current location.

[0815] "Target product information" is information about a specific product or product category that the user wants to find in the store.

[0816] "In-store conditions" refers to information about the structure and layout of the store, and conditions for route selection (e.g., passable aisles, product display positions).

[0817] "Calculate route" is a process of calculating the optimal route for the user to reach the target product based on current location information, target product information, and in-store conditions.

[0818] "Haptic feedback data" is data that a device uses to communicate information to a user through the sense of touch, specifically including sensory signals such as vibration and pressure.

[0819] A "correction route" is a new path that the user can take to get back on the correct route if they take a wrong turn.

[0820] The "arrival notification" is a notification accompanied by haptic feedback to inform the user that they have reached the desired product.

[0821] To realize this invention, a system that links the functions of a user terminal (such as a smartphone), a server, and the user is required. This system uses haptic feedback to help users find the product they are looking for in a physical store.

[0822] System Configuration

[0823] User terminal

[0824] The user terminal is a device that the user can carry around, such as a smartphone or a dedicated device. It includes location information acquisition functions such as GPS or beacons (Bluetooth) and vibration motors to provide haptic feedback. The user terminal communicates with the server via the Internet or Bluetooth.

[0825] server

[0826] The server uses cloud services such as Amazon Web Services (AWS). Platforms such as AWS EC2 and AWS Lambda are used to execute the series of processes. MySQL is also used as the database. The server receives the user's location information and desired product information, and uses a generative AI model (such as OpenAI's GPT-3) to calculate the optimal route.

[0827] Program processing

[0828] 1. User registration and initial settings

[0829] A user installs and launches the app for the first time, and enters their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[0830] This information is encrypted by the device and sent to an AWS server, which stores it in a database and generates a unique identification number for the user.

[0831] 2. In-store location and routing

[0832] When a user enters a store, the device obtains its current location using GPS or a beacon system.

[0833] Users use the app to input the desired product name or category, such as "Milk section."

[0834] The server receives this information and uses a generative AI model to calculate the optimal path and generate haptic feedback data.

[0835] 3. Real-time route guidance

[0836] Based on the haptic feedback data sent from the server, the device uses vibration patterns to provide guidance to the user.

[0837] If the user takes a wrong turn, the device calculates a corrective path and provides new haptic feedback data.

[0838] When the user arrives at the desired product, they are notified of the arrival via haptic feedback.

[0839] Specific examples

[0840] For example, if a user selects the "Milk Corner" as their destination, the app retrieves their current location and calculates the optimal route. Using a generative AI model, the app presents prompts that suggest the clearest and safest route for the user. If the user takes a wrong turn, the server calculates a new route and the device corrects the guidance with haptic feedback.

[0841] Example prompt sentence:

[0842] Japanese

[0843] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[0844] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0845] Step 1:

[0846] User registration and initial settings

[0847] Users install and launch the app, then enter their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[0848] The terminal encrypts this input information and transmits it to the server.

[0849] The server stores the received information in a MySQL database, generates a unique identification number for the user, and sends it back to the terminal.

[0850] Input: User information (name, age, needs).

[0851] Output: A unique identification number for the user.

[0852] Step 2:

[0853] Obtaining location information within the store

[0854] A user enters a store.

[0855] The device obtains its current location information using GPS or a beacon system (Bluetooth).

[0856] Once the current location information is obtained, it is sent to the server.

[0857] Input: None.

[0858] Output: Current location of the user.

[0859] Step 3:

[0860] Enter the desired product information

[0861] Users enter the desired product name or category into the app, such as "Milk section."

[0862] The terminal transmits the input target product information to the server.

[0863] Input: Desired product information.

[0864] Output: None.

[0865] Step 4:

[0866] Optimal Route Calculation

[0867] Based on the received current location information and desired product information, the server calculates the optimal route using a generative AI model (e.g., OpenAI GPT-3) taking into account in-store conditions.

[0868] The server generates the calculated optimal path as haptic feedback data and transmits it to the terminal.

[0869] Input: current location information, desired product information, in-store conditions.

[0870] Output: Haptic feedback data.

[0871] Step 5:

[0872] Providing haptic feedback

[0873] The device generates a vibration pattern based on the received haptic feedback data and provides guidance to the user through haptic feedback.

[0874] Input: Haptic feedback data.

[0875] Output: Haptic feedback via vibration.

[0876] Step 6:

[0877] Check and correct heading

[0878] If the user takes a wrong turn, the device will detect the heading error based on data from beacons or GPS.

[0879] If the terminal detects an error, it sends the information to the server.

[0880] The server calculates the corrective path and generates new haptic feedback data to send to the terminal.

[0881] Input: Heading error information.

[0882] Output: New haptic feedback data.

[0883] Step 7:

[0884] Confirmation that the desired product has arrived

[0885] When the user reaches the desired product, the terminal confirms arrival based on the location information.

[0886] The server generates data to notify the device of the arrival using haptic feedback and sends it to the device.

[0887] Input: Arrival location information.

[0888] Output: Reach haptic feedback data.

[0889] For example, here is the prompt again:

[0890] Japanese

[0891] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[0892] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0893] The present invention is a system that recognizes a user's emotions and provides adaptive haptic feedback. This system is composed of a user terminal, a server, and a user, and has an emotion engine built in.

[0894] System Configuration

[0895] The user terminal (smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[0896] User registration and initial settings

[0897] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[0898] Route learning and configuration

[0899] The user inputs their destination, and the device sends their current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past travel history, and specific requests. Haptic feedback data is generated and sent to the device. The device stores this data and provides haptic feedback when needed.

[0900] Emotion Recognition and Adaptive Feedback

[0901] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or tense, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[0902] Specific examples

[0903] When Elderly Person A goes to the hospital

[0904] Elderly user A starts the device and requests route guidance to the hospital. The device acquires current location information and sends destination information to the server. The server generates an optimal route and sends haptic feedback data to the device. The device provides appropriate haptic feedback to Elderly User A. If Elderly User A goes in the wrong direction, the device provides warning haptic feedback and suggests the correct direction.

[0905] If the device detects that Elderly A is feeling stressed or tense during the interaction, the device's emotion engine sends that information to the server, which can then adjust the feedback pattern and resend the haptic feedback to provide a sense of security.

[0906] If your child goes to school

[0907] A parent sets their child's school route on the device. As the child heads to school, the device tracks their progress in real time and provides route guidance. The server periodically receives location information to check whether the user is heading in the correct direction. If the user heads in the wrong direction, the server calculates a corrected route and sends new haptic feedback data to the device, which then provides a warning.

[0908] If the child feels tired during the process, the emotion engine sends that information to the server, which then provides haptic feedback to encourage them to take a break.

[0909] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[0910] The processing flow will be explained below.

[0911] Program processing details

[0912] User registration and initial settings

[0913] Step 1:

[0914] The user starts up the device and enters basic information such as their name, age, email address, and password on the login screen.

[0915] Step 2:

[0916] The terminal encrypts the basic information entered and transmits it to the server via the network.

[0917] Step 3:

[0918] The server stores the received basic information in the database and generates a new user ID.

[0919] Step 4:

[0920] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[0921] Step 5:

[0922] The device stores the received user ID and initializes the emotion engine to collect initial data to generate the user's emotion profile.

[0923] Route learning and configuration

[0924] Step 1:

[0925] The user operates the terminal to select a destination and start setting route guidance.

[0926] Step 2:

[0927] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[0928] Step 3:

[0929] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[0930] Step 4:

[0931] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[0932] Step 5:

[0933] The terminal stores the received haptic feedback data and provides it to the user as needed.

[0934] Emotion Recognition and Adaptive Feedback

[0935] Step 1:

[0936] Guidance will begin along the route specified by the user on the terminal.

[0937] Step 2:

[0938] The device periodically acquires GPS data and tracks its current location.

[0939] Step 3:

[0940] The device recognizes the user's emotions using sensor data such as voice, facial expressions, and heart rate.

[0941] Step 4:

[0942] The device transmits the recognized emotion data to the server.

[0943] Step 5:

[0944] The server evaluates the progress and emotion data based on the current location, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[0945] Step 6:

[0946] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path, generates new haptic feedback data, and sends it to the terminal.

[0947] Step 7:

[0948] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern: if the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[0949] Step 8:

[0950] The device adjusts the feedback pattern according to the user's emotions, providing haptic feedback to relieve stress and tension, while simultaneously providing haptic feedback to encourage rest if the user's emotions indicate fatigue.

[0951] Step 9:

[0952] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[0953] Specific examples

[0954] When Elderly Person A goes to the hospital

[0955] Step 1:

[0956] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[0957] Step 2:

[0958] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[0959] Step 3:

[0960] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[0961] Step 4:

[0962] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[0963] Step 5:

[0964] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[0965] Step 6:

[0966] The device recognizes the emotions of elderly person A using voice, facial expression, and heart rate sensors, and transmits the emotional data to the server.

[0967] Step 7:

[0968] The server adjusts the feedback pattern based on the received emotional data, regenerates haptic feedback that provides a sense of security, and sends it to the device.

[0969] Step 8:

[0970] When Elderly A reaches the hospital, the server confirms the arrival and sends a completion notification to the terminal, which then notifies Elderly A of the arrival via haptic feedback.

[0971] If your child goes to school

[0972] Step 1:

[0973] Parents can register their children's routes to school in advance on their devices.

[0974] Step 2:

[0975] As a child walks to school, the device tracks their progress in real time and provides route guidance.

[0976] Step 3:

[0977] The server periodically receives location information and checks whether the user is heading in the correct direction.

[0978] Step 4:

[0979] If a wrong turn occurs, the server calculates a corrective path and sends new haptic feedback data to the device.

[0980] Step 5:

[0981] The device will guide you in the right direction based on the new haptic feedback it receives, and adjust the vibration pattern to a more reassuring one if the user is feeling stressed or tense.

[0982] Step 6:

[0983] If the device senses fatigue, it will provide haptic feedback to encourage you to take a break.

[0984] Step 7:

[0985] When the child arrives at school, the server confirms the arrival, sends a notification of arrival to the terminal, and notifies the parents.

[0986] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[0987] Example 2

[0988] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0989] Conventional route guidance systems can obtain a user's current location information and provide the optimal route to a destination, but they do not provide feedback that takes into account the user's emotional state. In particular, there is a lack of means to help users with special needs, such as the elderly and children, safely reach their destination while reducing stress and tension. Furthermore, there is a need for a system that is flexible enough to provide appropriate real-time feedback regarding incorrect directions.

[0990] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and requirements specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and requirements and generating haptic feedback data using a generative AI model, means for providing the generated haptic feedback data to the user together with haptic feedback, means for collecting sensor data and recognizing the user's emotions in real time, and means for adjusting the haptic feedback pattern according to the user's emotions. This not only enables the user to reach their destination safely, but also makes it possible to provide appropriate feedback in real time according to errors in heading or the user's emotional state.

[0991] "Means for obtaining the user's current location information" refers to a function for identifying and obtaining the user's current geographical location using a GPS module or other location information services from the device the user is using.

[0992] The "means for obtaining user-specified destination information and requirements" refers to a function for collecting the specific destination and the user's specific wishes and requests that the user has input into the system.

[0993] "Means for calculating the optimal route and generating haptic feedback data using a generative AI model" refers to a function that uses an AI model to calculate the optimal route for the user based on the acquired current location information, destination information, and requirements, and creates data for haptic feedback.

[0994] The "means for providing the generated haptic feedback data to the user along with haptic feedback" is a function for using the generated haptic feedback data to provide a haptic notification to the user's device.

[0995] "Means for collecting sensor data and recognizing user emotions in real time" refers to a function that uses sensors installed in the user's device to collect and analyze data such as voice, facial expressions, and heart rate, in order to recognize the user's emotional state in real time.

[0996] The "means for adjusting the haptic feedback pattern in accordance with the user's emotion" is a function for appropriately changing the intensity and pattern of the haptic feedback based on the recognized emotion of the user.

[0997] MODE FOR CARRYING OUT THE INVENTION

[0998] The present invention provides a system for recognizing a user's emotions and adaptively providing haptic feedback. The system is comprised of a user terminal, a server, and a user, and has an emotion engine built in. Specific embodiments of the system are described below.

[0999] System Configuration

[1000] The user terminal (e.g., a smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[1001] Specific functions and operations

[1002] User registration and initial settings

[1003] 1. The user starts the device and enters basic information (name, age, specific requests).

[1004] 2. The device encrypts the entered information and sends it to the server.

[1005] 3. The server stores the information in a database, generates a user ID, and sends it to the device.

[1006] 4. The device collects initial data such as voice samples and heart rate to create an emotional profile.

[1007] Route learning and configuration

[1008] 1. The user enters the destination, and the device obtains the current location information and sends it to the server.

[1009] 2. The server uses a generative AI model to calculate the optimal route based on the current location, destination, past travel history, and specific requests.

[1010] 3. The server generates haptic feedback data and sends it to the device.

[1011] 4. The device stores the received data and provides haptic feedback when necessary.

[1012] Emotion Recognition and Adaptive Feedback

[1013] 1. The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate.

[1014] 2. The device analyzes the data collected and recognizes the user's emotions.

[1015] 3. The device sends the recognition results to the server.

[1016] 4. The server adjusts the haptic feedback pattern according to the user's emotional state and sends it back to the device.

[1017] 5. The device applies the new feedback pattern and provides haptic feedback to the user.

[1018] Specific examples

[1019] Example 1: Elderly person A goes to the hospital

[1020] 1. The user (elderly person A) starts up the device and requests "route guidance to the hospital."

[1021] 2. The device obtains the current location information using the GPS module and sends the destination information (hospital) to the server.

[1022] 3. The server generates the optimal path and sends haptic feedback data to the device.

[1023] 4. The device displays route directions and provides haptic feedback.

[1024] 5. If Elderly Person A goes in the wrong direction, the device will generate haptic feedback to warn him and show him the correct direction to go.

[1025] 6. If Elderly Person A feels stressed during the exercise, the device's emotion engine detects an increase in heart rate and sends the emotion data to the server.

[1026] 7. The server calculates reassuring haptic feedback and sends it to the device.

[1027] 8. The device provides new haptic feedback to reassure Elderly A.

[1028] Example 2: When a child goes to school

[1029] 1. Parents set their child's school route on the device.

[1030] 2. When the child starts going to school, the device tracks their current location and sends it to the server.

[1031] 3. The server periodically checks the location information and monitors the direction of travel.

[1032] 4. If the child takes a wrong turn, the server calculates a corrective path and sends new haptic feedback data to the device.

[1033] 5. The device will provide warnings and give you precise directions to follow.

[1034] 6. The device's emotion engine detects the child's fatigue and sends the data to the server.

[1035] 7. The server provides haptic feedback to encourage a break, and the device notifies the child.

[1036] Prompt Sentence Examples

[1037] Route guidance request

[1038] The user types into the terminal, "I want to go to the hospital. Please tell me the best route."

[1039] Emotional state report

[1040] The user types into the device, "I'm feeling a little stressed right now. Please provide some haptic feedback to help me relax."

[1041] As described above, this system recognizes the user's emotions in real time and provides optimal haptic feedback, allowing the user to reach their destination with peace of mind. Furthermore, by providing appropriate feedback in real time depending on the user's emotional state and direction, the system offers a safer and more comfortable travel experience.

[1042] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1043] Step 1: User registration and initial setup

[1044] 1. Start the device and enter basic information

[1045] The user turns on the device and enters basic information such as name, age and specific requests.

[1046] Input: Basic information (name, age, specific requests)

[1047] The terminal encrypts the information entered.

[1048] Output: Encrypted basic information

[1049] 2. Transmission and storage of information

[1050] The device sends the encrypted information to the server.

[1051] Input: Encrypted basic information

[1052] The server stores the received information in a database and generates a user ID.

[1053] Output: User ID

[1054] The server sends the generated user ID to the terminal.

[1055] 3. Initial Data Collection

[1056] The device collects initial data such as voice samples and heart rate to create an emotional profile of the user.

[1057] Input: User voice samples, heart rate, etc.

[1058] The device sends the collected data to a server to create an emotional profile.

[1059] Output: Emotion profile

[1060] Step 2: Route learning and configuration

[1061] 1. Set your destination

[1062] The user inputs a destination into the terminal.

[1063] Input: Destination information

[1064] The device acquires the current location information and sends it to the server.

[1065] Input: Current location information

[1066] Output: Current location information, destination information

[1067] 2. Route calculation

[1068] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past travel history, and specific requests.

[1069] Input: current location, destination, past travel history, specific requests

[1070] Output: Optimal route data

[1071] The server generates haptic feedback data based on the calculation results.

[1072] Output: Haptic feedback data

[1073] 3. Sending and storing feedback data

[1074] The server transmits the generated haptic feedback data to the terminal.

[1075] Input: Haptic feedback data

[1076] The device stores the received data and provides haptic feedback if necessary.

[1077] Output: Stored haptic feedback data

[1078] Step 3: Emotion recognition and adaptive feedback

[1079] 1. Sensor data collection

[1080] The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate in real time.

[1081] Input: Voice data, facial expression data, heart rate data

[1082] Output: Collected sensor data

[1083] 2. Emotional Recognition

[1084] The device's emotion engine analyzes the collected data and recognizes the user's emotions.

[1085] Input: Collected sensor data

[1086] Output: Emotional state data

[1087] The device sends the analysis results to the server.

[1088] 3. Adjusting feedback patterns

[1089] The server optimizes the haptic feedback pattern based on the user's emotional state.

[1090] Input: Emotional state data

[1091] Output: Tuned feedback pattern

[1092] The server retransmits the adjusted feedback pattern to the terminal.

[1093] The device applies the new feedback pattern and provides haptic feedback to the user.

[1094] (Application example 2)

[1095] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1096] In modern shopping experiences, users are often overwhelmed by a large amount of information, which can lead to feelings of stress and anxiety. However, there is a lack of technology that can recognize these emotions in real time and respond appropriately. In particular, virtual stores are required to provide appropriate feedback based on the user's emotional state, but no effective system exists for this purpose.

[1097] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1098] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and route conditions specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and route conditions and generating haptic feedback data, means for recognizing the user's emotions in real time and transmitting the data to the server, and means for receiving an optimal haptic feedback pattern from the server and providing it to the user. This makes it possible to reduce the stress and anxiety felt by the user during their shopping experience in a virtual store and provide appropriate haptic feedback and audio feedback.

[1099] "Current location information" is data indicating the location where the user is currently located.

[1100] "Destination information" is data indicating the location the user is trying to reach.

[1101] "Route conditions" are data indicating restrictions and requests regarding the route specified by the user.

[1102] "Haptic feedback data" is data that provides a physical response to the user's senses.

[1103] "Means for real-time emotion recognition" refers to a technical mechanism for instantly determining a user's current emotional state.

[1104] A "server" is a computer system that processes and stores data over a network.

[1105] "Feedback based on emotional state" refers to tactile, audio, or other responses provided in response to the user's emotions.

[1106] A "virtual store" is a virtual shopping environment created on the Internet.

[1107] "Haptic feedback" is a technology that conveys information to a user through physical sensation.

[1108] The present invention provides a system that recognizes a user's emotions in real time and provides adaptive haptic feedback. This system is comprised of a user terminal, a server, and a user, and has an emotion engine built in.

[1109] System Configuration

[1110] The user terminal (smart glasses or smartphone) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time to generate optimal haptic feedback.

[1111] User registration and initial settings

[1112] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[1113] Route learning and configuration

[1114] When a user starts shopping in a virtual store, the device sends its current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past behavior history, and specific requests. It generates haptic feedback data and sends it to the device. The device stores this data and provides haptic feedback when needed.

[1115] Emotion Recognition and Adaptive Feedback

[1116] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or anxious, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[1117] Specific examples

[1118] A user wears smart glasses and visits a virtual store. While searching for a product in the store, the emotion engine detects the user's anxiety and sends the emotion data to the server. The server calculates the optimal haptic feedback pattern and sends it to the device. The device then provides vibration and audio feedback, reducing the user's stress.

[1119] Prompt Sentence Examples

[1120] It recognizes the user's emotions while selecting products in a virtual store and provides immediate relaxation feedback.

[1121] It detects the stress felt by users searching for a specific product in a virtual store and provides haptic feedback.

[1122] This system and method provide optimal haptic feedback based on the user's emotional state, enabling a comfortable shopping experience in a virtual store.

[1123] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1124] Step 1:

[1125] The user turns on the device and enters basic information (name, age, specific requests).

[1126] Specific operation: The user enters their name, age, and specific requests into the terminal, and then sends that information to the server by pressing the send button.

[1127] Input: User's basic information (name, age, specific needs).

[1128] Data processing / calculation: The basic information entered by the terminal is encrypted and sent to the server.

[1129] Output: Encrypted user data.

[1130] Step 2:

[1131] The server stores the received user information in a database, generates a user ID, and sends it to the terminal.

[1132] Specific operation: The server saves the received user information in a database, generates a new user ID, and returns the ID to the device.

[1133] Input: Encrypted user data.

[1134] Data processing / calculation: Saving user data and generating user IDs.

[1135] Output: The generated user ID.

[1136] Step 3:

[1137] The device collects initial data to generate an emotional profile of the user.

[1138] Specific operation: The device records initial emotion data using the camera, microphone, heart rate sensor, etc.

[1139] Input: User's voice, facial expressions, heart rate, etc.

[1140] Data processing / computation: Collecting and recording emotional data.

[1141] Output: Initial emotion profile data.

[1142] Step 4:

[1143] When a user starts shopping in a virtual store, the terminal transmits current location information to the server.

[1144] Specific operation: The device uses the GPS module to obtain its current location information and sends it to the server.

[1145] Input: The user's current location.

[1146] Data processing / calculation: Acquisition and transmission of location information.

[1147] Output: Current location information sent to the server.

[1148] Step 5:

[1149] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past behavior, and specific requests.

[1150] Specific operation: Based on the data received by the server, the generative AI model is used to calculate the optimal shopping route.

[1151] Input: current location, destination, past activity history, specific requests.

[1152] Data processing / calculation: Route calculation using generative AI models.

[1153] Output: Optimal route data.

[1154] Step 6:

[1155] Haptic feedback data is generated based on the optimal path and transmitted to the device.

[1156] Specific operation: The server generates haptic feedback data based on the route data calculated and sends it to the device.

[1157] Input: Optimal route data.

[1158] Data processing / computation: Generation of haptic feedback data.

[1159] Output: Haptic feedback data.

[1160] Step 7:

[1161] The terminal stores the received haptic feedback data and provides haptic feedback when necessary.

[1162] Specific operation: The device stores the received data in its internal storage and provides it to the user via the haptic feedback device when necessary.

[1163] Input: Haptic feedback data.

[1164] Data processing / calculation: Storing data and giving instructions to the haptic device.

[1165] Output: Haptic feedback.

[1166] Step 8:

[1167] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate.

[1168] Specific operation: The device inputs data acquired by sensors into the emotion engine and analyzes the user's emotions in real time.

[1169] Input: speech, facial expression, and heart rate data.

[1170] Data processing / computation: Analysis of emotion data.

[1171] Output: User's emotional state data.

[1172] Step 9:

[1173] If the user is feeling stressed or anxious, the device sends that information to the server.

[1174] Specific operation: The terminal transmits the user's emotional state to the server, and the server receives it.

[1175] Input: Emotional state data.

[1176] Data processing / calculation: Sending and receiving emotional data.

[1177] Output: Emotion data sent to the server.

[1178] Step 10:

[1179] The server calculates the optimal haptic feedback pattern and sends it to the device.

[1180] Specific operation: Based on the emotional state, the server adjusts the haptic feedback, generates new patterns, and sends them to the device.

[1181] Input: Emotional state data.

[1182] Data processing / calculation: Generation of optimal haptic feedback patterns.

[1183] Output: New haptic feedback data.

[1184] Step 11:

[1185] The device provides new haptic feedback to the user, reducing stress and anxiety.

[1186] Specific operation: The device activates the haptic feedback device based on the new haptic feedback data, providing the user with a sense of security.

[1187] Input: New haptic feedback data.

[1188] Data processing / computation: Implementing haptic feedback.

[1189] Output: Improved user emotional state.

[1190] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1191] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1192] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1193] [Third embodiment]

[1194] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1195] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1196] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1197] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1198] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1199] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1200] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1201] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1202] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1203] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1204] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1205] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1206] The present invention provides a system for safely guiding a user to a destination using haptic feedback. This allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own. An embodiment of the present invention will be described below.

[1207] System Configuration

[1208] This system consists of a user terminal (a smartphone or dedicated device), a server connected to it, and the user. Communication between each component is carried out via a network (e.g., the Internet).

[1209] Program processing

[1210] User registration and initial settings

[1211] When a user uses the system for the first time, they start up their device, log in, and enter basic information (name, age, specific requests). The device encrypts this information and sends it to the server. The server stores the received information in a database, generates a unique user identification number, and sends it back to the device. This makes it possible to manage the user's individual data and movement history.

[1212] Route learning and configuration

[1213] When a user heads to a specific destination, they input destination information from their device. The device acquires GPS data and sends it to the server along with their current location. Based on this information, the server uses a generative AI model to calculate the optimal route. This optimal route takes into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes). The calculated route is generated as haptic feedback data and sent to the device. The device stores this haptic feedback data and uses it when necessary.

[1214] Specific examples of real-time guidance

[1215] When Elderly Person A goes to the hospital

[1216] Elderly user A holds the device and requests route guidance to the hospital. The device acquires its current location information and sends it to the server with the hospital as the destination. The server generates an optimal route based on past movement patterns (e.g., a route with few slopes) and sends this as haptic feedback data to the device. The device then uses the haptic data to guide Elderly User A using a vibration pattern. If Elderly User A goes in the wrong direction, the device will warn them with a different vibration pattern and show them the correct direction. Finally, the server confirms that Elderly User A has reached the hospital and notifies them of the completion using haptic feedback.

[1217] If your child goes to school

[1218] A child's parent pre-registers their child's route to school on the device. As the child carries the device to school, the device tracks their progress in real time and provides guidance. The server periodically receives location information and checks whether the child is progressing correctly. If the child takes a wrong turn, the server calculates a corrected route and sends new haptic feedback data. The device then provides haptic feedback accordingly, correcting the route. When the child arrives at school, a notification is sent to the parent, providing peace of mind.

[1219] The above is an embodiment of the present invention, which is a system that helps various users reach their destination safely and easily. This system provides route guidance through tactile feedback without relying on vision, so anyone can easily use it.

[1220] The processing flow will be explained below.

[1221] Program processing details

[1222] User registration and initial settings

[1223] Step 1:

[1224] The user starts up the device and enters basic information such as their name, age, email address, and password into the application's login screen.

[1225] Step 2:

[1226] The terminal encrypts the basic information entered and transmits it to the server via the network.

[1227] Step 3:

[1228] The server stores the received basic information in the database and generates a new user ID.

[1229] Step 4:

[1230] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[1231] Step 5:

[1232] The device saves the received user ID and completes the initial setup.

[1233] Route learning and configuration

[1234] Step 1:

[1235] The user operates the terminal to select a destination and start setting route guidance.

[1236] Step 2:

[1237] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[1238] Step 3:

[1239] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[1240] Step 4:

[1241] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[1242] Step 5:

[1243] The terminal stores the received haptic feedback data and provides it to the user as needed.

[1244] Real-time guidance

[1245] Step 1:

[1246] To start guidance along the route specified by the user, route guidance is started on the terminal.

[1247] Step 2:

[1248] The device periodically acquires GPS data and tracks its current location.

[1249] Step 3:

[1250] The device sends its current location information to the server.

[1251] Step 4:

[1252] The server evaluates the progress based on the current position, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[1253] Step 5:

[1254] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path and generates new haptic feedback data to send to the device.

[1255] Step 6:

[1256] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern. If the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[1257] Step 7:

[1258] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[1259] Specific examples

[1260] When Elderly Person A goes to the hospital

[1261] Step 1:

[1262] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[1263] Step 2:

[1264] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[1265] Step 3:

[1266] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[1267] Step 4:

[1268] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[1269] Step 5:

[1270] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[1271] Step 6:

[1272] When Elderly A reaches his destination, the hospital, the server confirms his arrival and sends a completion notification to the terminal, which then notifies Elderly A via haptic feedback.

[1273] In this way, the system can effectively guide the user to their destination using haptic feedback.

[1274] Example 1

[1275] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1276] To provide a system that enables the visually impaired, the elderly, children, etc. to safely reach their destinations on their own. In particular, to improve the safety and convenience of users by enabling them to receive route guidance without relying on visual information and by providing appropriate guidance even when heading in the wrong direction.

[1277] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1278] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and travel conditions specified by the user, means for calculating an optimal route using a generative AI model based on the acquired current location information, destination information, and travel conditions, and means for generating haptic feedback data, and means for generating new haptic feedback data. This enables anyone to easily and safely reach their destination without relying on vision.

[1279] A "user terminal" is an electronic device carried by users such as visually impaired people, the elderly, and children, and is primarily used to obtain GPS data and provide tactile feedback.

[1280] The "server" is a central computer system that receives data sent from user terminals, calculates optimal routes, generates haptic feedback data, and stores the user's movement history.

[1281] "Current location information" is real-time location data of a user obtained using the GPS function of the user terminal.

[1282] "Destination information" refers to geographical information of a place that a user wants to reach, which is input into the terminal.

[1283] "Travel conditions" is information including specific requests and constraints specified by the user (e.g., avoiding slopes, width of sidewalks, etc.).

[1284] A "generative AI model" is an artificial intelligence technology used by the server, and is an algorithm that calculates the optimal route based on past data and user characteristics.

[1285] "Haptic feedback data" is data that is sent to a user terminal and is used to provide route guidance to the user in the form of vibration patterns or the like.

[1286] A "corrected route" is a new optimal route that is recalculated when the user takes a wrong turn.

[1287] "Arrival confirmation" is the process by which the server confirms that the user has reached their destination.

[1288] "Haptic feedback" is a means of physically conveying route guidance and warnings to users using the device's vibration function, etc.

[1289] System Overview

[1290] The present invention is a system for safely guiding a user to a destination using haptic feedback, which allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own.

[1291] System Configuration

[1292] The system consists of the following components:

[1293] 1. User device (e.g., smartphone or dedicated device)

[1294] 2. Server

[1295] 3. Users

[1296] Communication takes place over the Internet.

[1297] Program processing

[1298] User registration and initial settings

[1299] When a user starts up the device for the first time, they are taken to a login screen where they enter basic information such as their name, age, and any specific requests.

[1300] The terminal uses AES (Advanced Encryption Standard) to encrypt the basic information entered and sends this encrypted data to the server.

[1301] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[1302] Route learning and configuration

[1303] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[1304] The device uses a GPS module to obtain data on its current location and transmits this information to a server.

[1305] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes).

[1306] The calculated path is generated as haptic feedback data, which is sent to the device, which stores the haptic feedback data in its internal memory and uses it as needed.

[1307] Real-time guidance

[1308] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[1309] The server periodically receives the user's location information and verifies that the route is correct.

[1310] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[1311] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[1312] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data to the device as a completion notification. This feedback data conveys the end notification to the user.

[1313] Specific examples

[1314] When Elderly Person A goes to the hospital

[1315] 1. The user, elderly person A, takes the device and requests route guidance to the hospital.

[1316] 2. The device obtains its current location information using the GPS module and sends it to the server with the hospital as the destination.

[1317] 3. The server generates the optimal route based on past travel patterns and basic information.

[1318] 4. The device guides Elderly A using vibration patterns based on the haptic feedback data. For example, a right turn is indicated by three short vibrations, and a left turn is indicated by one long vibration.

[1319] 5. If Elderly Person A goes in the wrong direction, the device will warn him with five short vibrations and provide him with a recalculated correct route.

[1320] 6. The server confirms that the delivery has reached the hospital and notifies the completion with a "3-second continuous vibration."

[1321] If your child goes to school

[1322] 1. Parents register their child's route to school on the device in advance.

[1323] 2. When a child carries a device and heads to school, the device tracks their location in real time.

[1324] 3. Periodically send location information to the server to confirm correct progress.

[1325] 4. If the child takes a wrong turn, the server instantly calculates a corrective path and sends a new vibration pattern to the device.

[1326] 5. When the child reaches the destination (school), an "arrival notification" is sent to the parent, providing peace of mind to the parent.

[1327] Examples of prompt statements

[1328] "Generate route guidance for an elderly person going to the hospital based on the following information: Name - Ichiro Suzuki, Age - 70, Current location - {Current location coordinates}, Destination - {Hospital coordinates}, Past travel patterns - {Past travel history}"

[1329] This provides a system that helps users reach their destination safely and easily.

[1330] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1331] Program processing steps

[1332] User registration and initial settings

[1333] Step 1:

[1334] A user starts up the device for the first time and is directed to a login screen, where they enter basic information such as their name, age, and any specific requests.

[1335] Input: Name, age, specific requests

[1336] Output: Basic information data

[1337] Step 2:

[1338] The terminal acquires the basic information entered and encrypts it using AES (Advanced Encryption Standard).

[1339] Input: Basic information data

[1340] Output: Encrypted basic information data

[1341] Step 3:

[1342] The terminal transmits the encrypted basic information data to the server.

[1343] Input: Encrypted basic information data

[1344] Output: None (data transmission completed)

[1345] Step 4:

[1346] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[1347] Input: Encrypted basic information data

[1348] Output: A unique identification number for the user

[1349] Route learning and configuration

[1350] Step 5:

[1351] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[1352] Input: Destination information

[1353] Output: None (Destination information input complete)

[1354] Step 6:

[1355] The device uses a GPS module to obtain data on its current location and sends this information along with basic and destination information to a server.

[1356] Input: GPS data, basic information, destination information

[1357] Output: None (data transmission completed)

[1358] Step 7:

[1359] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests.

[1360] Input: GPS data, basic information, destination information, past travel history, specific requests

[1361] Output: Optimal route data

[1362] Step 8:

[1363] The server generates the calculated optimal route data as haptic feedback data and transmits this data to the terminal.

[1364] Input: Optimal route data

[1365] Output: Haptic feedback data

[1366] Step 9:

[1367] The device stores the received haptic feedback data in its internal memory and uses it as needed.

[1368] Input: Haptic feedback data

[1369] Output: None (data saving completed)

[1370] Real-time guidance

[1371] Step 10:

[1372] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[1373] Input: None (real-time location acquisition)

[1374] Output: Current location information

[1375] Step 11:

[1376] The server periodically receives the user's location information and verifies that the route is correct.

[1377] Input: Current location information

[1378] Output: Correct route confirmation result

[1379] Step 12:

[1380] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[1381] Input: haptic feedback data, current location information

[1382] Output: Vibration pattern

[1383] Step 13:

[1384] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[1385] Input: current location, inaccurate heading

[1386] Output: New haptic feedback data, vibration patterns

[1387] Step 14:

[1388] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data as a completion notification to the device, which then receives the data and notifies the user.

[1389] Input: Confirmation of arrival at destination

[1390] Output: Haptic feedback data and vibration pattern for completion notification

[1391] This allows the user to reach their destination safely and easily.

[1392] (Application example 1)

[1393] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1394] People with visual impairments, the elderly, and children face challenges in finding the products they are looking for in physical stores. Navigating safely and efficiently without relying on vision is difficult. Furthermore, there is a lack of systems that provide appropriate feedback to users when they have taken a wrong turn or when they have reached the desired product. This makes in-store shopping stressful and inconvenient for many users.

[1395] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1396] In this invention, the server includes means for acquiring user's current location information, means for acquiring user-specified target product information and in-store conditions, and means for calculating an optimal route based on the acquired current location information, target product information, and in-store conditions, and generating haptic feedback data, thereby enabling visually impaired people, the elderly, and children to move safely and efficiently within a physical store and reach their target products.

[1397] "Current location information" is data for identifying the user's current location.

[1398] "Target product information" is information about a specific product or product category that the user wants to find in the store.

[1399] "In-store conditions" refers to information about the structure and layout of the store, and conditions for route selection (e.g., passable aisles, product display positions).

[1400] "Calculate route" is a process of calculating the optimal route for the user to reach the target product based on current location information, target product information, and in-store conditions.

[1401] "Haptic feedback data" is data that a device uses to communicate information to a user through the sense of touch, specifically including sensory signals such as vibration and pressure.

[1402] A "correction route" is a new path that the user can take to get back on the correct route if they take a wrong turn.

[1403] The "arrival notification" is a notification accompanied by haptic feedback to inform the user that they have reached the desired product.

[1404] To realize this invention, a system that links the functions of a user terminal (such as a smartphone), a server, and the user is required. This system uses haptic feedback to help users find the product they are looking for in a physical store.

[1405] System Configuration

[1406] User terminal

[1407] The user terminal is a device that the user can carry around, such as a smartphone or a dedicated device. It includes location information acquisition functions such as GPS or beacons (Bluetooth) and vibration motors to provide haptic feedback. The user terminal communicates with the server via the Internet or Bluetooth.

[1408] server

[1409] The server uses cloud services such as Amazon Web Services (AWS). Platforms such as AWS EC2 and AWS Lambda are used to execute the series of processes. MySQL is also used as the database. The server receives the user's location information and desired product information, and uses a generative AI model (such as OpenAI's GPT-3) to calculate the optimal route.

[1410] Program processing

[1411] 1. User registration and initial settings

[1412] A user installs and launches the app for the first time, and enters their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[1413] This information is encrypted by the device and sent to an AWS server, which stores it in a database and generates a unique identification number for the user.

[1414] 2. In-store location and routing

[1415] When a user enters a store, the device obtains its current location using GPS or a beacon system.

[1416] Users use the app to input the desired product name or category, such as "Milk section."

[1417] The server receives this information and uses a generative AI model to calculate the optimal path and generate haptic feedback data.

[1418] 3. Real-time route guidance

[1419] Based on the haptic feedback data sent from the server, the device uses vibration patterns to provide guidance to the user.

[1420] If the user takes a wrong turn, the device calculates a corrective path and provides new haptic feedback data.

[1421] When the user arrives at the desired product, they are notified of the arrival via haptic feedback.

[1422] Specific examples

[1423] For example, if a user selects the "Milk Corner" as their destination, the app retrieves their current location and calculates the optimal route. Using a generative AI model, the app presents prompts that suggest the clearest and safest route for the user. If the user takes a wrong turn, the server calculates a new route and the device corrects the guidance with haptic feedback.

[1424] Example prompt sentence:

[1425] Japanese

[1426] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[1427] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1428] Step 1:

[1429] User registration and initial settings

[1430] Users install and launch the app, then enter their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[1431] The terminal encrypts this input information and transmits it to the server.

[1432] The server stores the received information in a MySQL database, generates a unique identification number for the user, and sends it back to the terminal.

[1433] Input: User information (name, age, needs).

[1434] Output: A unique identification number for the user.

[1435] Step 2:

[1436] Obtaining location information within the store

[1437] A user enters a store.

[1438] The device obtains its current location information using GPS or a beacon system (Bluetooth).

[1439] Once the current location information is obtained, it is sent to the server.

[1440] Input: None.

[1441] Output: Current location of the user.

[1442] Step 3:

[1443] Enter the desired product information

[1444] Users enter the desired product name or category into the app, such as "Milk section."

[1445] The terminal transmits the input target product information to the server.

[1446] Input: Desired product information.

[1447] Output: None.

[1448] Step 4:

[1449] Optimal Route Calculation

[1450] Based on the received current location information and desired product information, the server calculates the optimal route using a generative AI model (e.g., OpenAI GPT-3) taking into account in-store conditions.

[1451] The server generates the calculated optimal path as haptic feedback data and transmits it to the terminal.

[1452] Input: current location information, desired product information, in-store conditions.

[1453] Output: Haptic feedback data.

[1454] Step 5:

[1455] Providing haptic feedback

[1456] The device generates a vibration pattern based on the received haptic feedback data and provides guidance to the user through haptic feedback.

[1457] Input: Haptic feedback data.

[1458] Output: Haptic feedback via vibration.

[1459] Step 6:

[1460] Check and correct heading

[1461] If the user takes a wrong turn, the device will detect the heading error based on data from beacons or GPS.

[1462] If the terminal detects an error, it sends the information to the server.

[1463] The server calculates the corrective path and generates new haptic feedback data to send to the terminal.

[1464] Input: Heading error information.

[1465] Output: New haptic feedback data.

[1466] Step 7:

[1467] Confirmation that the desired product has arrived

[1468] When the user reaches the desired product, the terminal confirms arrival based on the location information.

[1469] The server generates data to notify the device of the arrival using haptic feedback and sends it to the device.

[1470] Input: Arrival location information.

[1471] Output: Reach haptic feedback data.

[1472] For example, here is the prompt again:

[1473] Japanese

[1474] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[1475] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1476] The present invention is a system that recognizes a user's emotions and provides adaptive haptic feedback. This system is composed of a user terminal, a server, and a user, and has an emotion engine built in.

[1477] System Configuration

[1478] The user terminal (smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[1479] User registration and initial settings

[1480] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[1481] Route learning and configuration

[1482] The user inputs their destination, and the device sends their current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past travel history, and specific requests. Haptic feedback data is generated and sent to the device. The device stores this data and provides haptic feedback when needed.

[1483] Emotion Recognition and Adaptive Feedback

[1484] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or tense, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[1485] Specific examples

[1486] When Elderly Person A goes to the hospital

[1487] Elderly user A starts the device and requests route guidance to the hospital. The device acquires current location information and sends destination information to the server. The server generates an optimal route and sends haptic feedback data to the device. The device provides appropriate haptic feedback to Elderly User A. If Elderly User A goes in the wrong direction, the device provides warning haptic feedback and suggests the correct direction.

[1488] If the device detects that Elderly A is feeling stressed or tense during the interaction, the device's emotion engine sends that information to the server, which can then adjust the feedback pattern and resend the haptic feedback to provide a sense of security.

[1489] If your child goes to school

[1490] A parent sets their child's school route on the device. As the child heads to school, the device tracks their progress in real time and provides route guidance. The server periodically receives location information to check whether the user is heading in the correct direction. If the user heads in the wrong direction, the server calculates a corrected route and sends new haptic feedback data to the device, which then provides a warning.

[1491] If the child feels tired during the process, the emotion engine sends that information to the server, which then provides haptic feedback to encourage them to take a break.

[1492] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[1493] The processing flow will be explained below.

[1494] Program processing details

[1495] User registration and initial settings

[1496] Step 1:

[1497] The user starts up the device and enters basic information such as their name, age, email address, and password on the login screen.

[1498] Step 2:

[1499] The terminal encrypts the basic information entered and transmits it to the server via the network.

[1500] Step 3:

[1501] The server stores the received basic information in the database and generates a new user ID.

[1502] Step 4:

[1503] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[1504] Step 5:

[1505] The device stores the received user ID and initializes the emotion engine to collect initial data to generate the user's emotion profile.

[1506] Route learning and configuration

[1507] Step 1:

[1508] The user operates the terminal to select a destination and start setting route guidance.

[1509] Step 2:

[1510] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[1511] Step 3:

[1512] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[1513] Step 4:

[1514] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[1515] Step 5:

[1516] The terminal stores the received haptic feedback data and provides it to the user as needed.

[1517] Emotion Recognition and Adaptive Feedback

[1518] Step 1:

[1519] Guidance will begin along the route specified by the user on the terminal.

[1520] Step 2:

[1521] The device periodically acquires GPS data and tracks its current location.

[1522] Step 3:

[1523] The device recognizes the user's emotions using sensor data such as voice, facial expressions, and heart rate.

[1524] Step 4:

[1525] The device transmits the recognized emotion data to the server.

[1526] Step 5:

[1527] The server evaluates the progress and emotion data based on the current location, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[1528] Step 6:

[1529] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path, generates new haptic feedback data, and sends it to the terminal.

[1530] Step 7:

[1531] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern: if the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[1532] Step 8:

[1533] The device adjusts the feedback pattern according to the user's emotions, providing haptic feedback to relieve stress and tension, while simultaneously providing haptic feedback to encourage rest if the user's emotions indicate fatigue.

[1534] Step 9:

[1535] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[1536] Specific examples

[1537] When Elderly Person A goes to the hospital

[1538] Step 1:

[1539] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[1540] Step 2:

[1541] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[1542] Step 3:

[1543] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[1544] Step 4:

[1545] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[1546] Step 5:

[1547] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[1548] Step 6:

[1549] The device recognizes the emotions of elderly person A using voice, facial expression, and heart rate sensors, and transmits the emotional data to the server.

[1550] Step 7:

[1551] The server adjusts the feedback pattern based on the received emotional data, regenerates haptic feedback that provides a sense of security, and sends it to the device.

[1552] Step 8:

[1553] When Elderly A reaches the hospital, the server confirms the arrival and sends a completion notification to the terminal, which then notifies Elderly A of the arrival via haptic feedback.

[1554] If your child goes to school

[1555] Step 1:

[1556] Parents can register their children's routes to school in advance on their devices.

[1557] Step 2:

[1558] As a child walks to school, the device tracks their progress in real time and provides route guidance.

[1559] Step 3:

[1560] The server periodically receives location information and checks whether the user is heading in the correct direction.

[1561] Step 4:

[1562] If a wrong turn occurs, the server calculates a corrective path and sends new haptic feedback data to the device.

[1563] Step 5:

[1564] The device will guide you in the right direction based on the new haptic feedback it receives, and adjust the vibration pattern to a more reassuring one if the user is feeling stressed or tense.

[1565] Step 6:

[1566] If the device senses fatigue, it will provide haptic feedback to encourage you to take a break.

[1567] Step 7:

[1568] When the child arrives at school, the server confirms the arrival, sends a notification of arrival to the terminal, and notifies the parents.

[1569] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[1570] Example 2

[1571] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1572] Conventional route guidance systems can obtain a user's current location information and provide the optimal route to a destination, but they do not provide feedback that takes into account the user's emotional state. In particular, there is a lack of means to help users with special needs, such as the elderly and children, safely reach their destination while reducing stress and tension. Furthermore, there is a need for a system that is flexible enough to provide appropriate real-time feedback regarding incorrect directions.

[1573] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and requirements specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and requirements and generating haptic feedback data using a generative AI model, means for providing the generated haptic feedback data to the user together with haptic feedback, means for collecting sensor data and recognizing the user's emotions in real time, and means for adjusting the haptic feedback pattern according to the user's emotions. This not only enables the user to reach their destination safely, but also makes it possible to provide appropriate feedback in real time according to errors in heading or the user's emotional state.

[1574] "Means for obtaining the user's current location information" refers to a function for identifying and obtaining the user's current geographical location using a GPS module or other location information services from the device the user is using.

[1575] The "means for obtaining user-specified destination information and requirements" refers to a function for collecting the specific destination and the user's specific wishes and requests that the user has input into the system.

[1576] "Means for calculating the optimal route and generating haptic feedback data using a generative AI model" refers to a function that uses an AI model to calculate the optimal route for the user based on the acquired current location information, destination information, and requirements, and creates data for haptic feedback.

[1577] The "means for providing the generated haptic feedback data to the user along with haptic feedback" is a function for using the generated haptic feedback data to provide a haptic notification to the user's device.

[1578] "Means for collecting sensor data and recognizing user emotions in real time" refers to a function that uses sensors installed in the user's device to collect and analyze data such as voice, facial expressions, and heart rate, in order to recognize the user's emotional state in real time.

[1579] The "means for adjusting the haptic feedback pattern in accordance with the user's emotion" is a function for appropriately changing the intensity and pattern of the haptic feedback based on the recognized emotion of the user.

[1580] MODE FOR CARRYING OUT THE INVENTION

[1581] The present invention provides a system for recognizing a user's emotions and adaptively providing haptic feedback. The system is comprised of a user terminal, a server, and a user, and has an emotion engine built in. Specific embodiments of the system are described below.

[1582] System Configuration

[1583] The user terminal (e.g., a smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[1584] Specific functions and operations

[1585] User registration and initial settings

[1586] 1. The user starts the device and enters basic information (name, age, specific requests).

[1587] 2. The device encrypts the entered information and sends it to the server.

[1588] 3. The server stores the information in a database, generates a user ID, and sends it to the device.

[1589] 4. The device collects initial data such as voice samples and heart rate to create an emotional profile.

[1590] Route learning and configuration

[1591] 1. The user enters the destination, and the device obtains the current location information and sends it to the server.

[1592] 2. The server uses a generative AI model to calculate the optimal route based on the current location, destination, past travel history, and specific requests.

[1593] 3. The server generates haptic feedback data and sends it to the device.

[1594] 4. The device stores the received data and provides haptic feedback when necessary.

[1595] Emotion Recognition and Adaptive Feedback

[1596] 1. The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate.

[1597] 2. The device analyzes the data collected and recognizes the user's emotions.

[1598] 3. The device sends the recognition results to the server.

[1599] 4. The server adjusts the haptic feedback pattern according to the user's emotional state and sends it back to the device.

[1600] 5. The device applies the new feedback pattern and provides haptic feedback to the user.

[1601] Specific examples

[1602] Example 1: Elderly person A goes to the hospital

[1603] 1. The user (elderly person A) starts up the device and requests "route guidance to the hospital."

[1604] 2. The device obtains the current location information using the GPS module and sends the destination information (hospital) to the server.

[1605] 3. The server generates the optimal path and sends haptic feedback data to the device.

[1606] 4. The device displays route directions and provides haptic feedback.

[1607] 5. If Elderly Person A goes in the wrong direction, the device will generate haptic feedback to warn him and show him the correct direction to go.

[1608] 6. If Elderly Person A feels stressed during the exercise, the device's emotion engine detects an increase in heart rate and sends the emotion data to the server.

[1609] 7. The server calculates reassuring haptic feedback and sends it to the device.

[1610] 8. The device provides new haptic feedback to reassure Elderly A.

[1611] Example 2: When a child goes to school

[1612] 1. Parents set their child's school route on the device.

[1613] 2. When the child starts going to school, the device tracks their current location and sends it to the server.

[1614] 3. The server periodically checks the location information and monitors the direction of travel.

[1615] 4. If the child takes a wrong turn, the server calculates a corrective path and sends new haptic feedback data to the device.

[1616] 5. The device will provide warnings and give you precise directions to follow.

[1617] 6. The device's emotion engine detects the child's fatigue and sends the data to the server.

[1618] 7. The server provides haptic feedback to encourage a break, and the device notifies the child.

[1619] Prompt Sentence Examples

[1620] Route guidance request

[1621] The user types into the terminal, "I want to go to the hospital. Please tell me the best route."

[1622] Emotional state report

[1623] The user types into the device, "I'm feeling a little stressed right now. Please provide some haptic feedback to help me relax."

[1624] As described above, this system recognizes the user's emotions in real time and provides optimal haptic feedback, allowing the user to reach their destination with peace of mind. Furthermore, by providing appropriate feedback in real time depending on the user's emotional state and direction, the system offers a safer and more comfortable travel experience.

[1625] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1626] Step 1: User registration and initial setup

[1627] 1. Start the device and enter basic information

[1628] The user turns on the device and enters basic information such as name, age and specific requests.

[1629] Input: Basic information (name, age, specific requests)

[1630] The terminal encrypts the information entered.

[1631] Output: Encrypted basic information

[1632] 2. Transmission and storage of information

[1633] The device sends the encrypted information to the server.

[1634] Input: Encrypted basic information

[1635] The server stores the received information in a database and generates a user ID.

[1636] Output: User ID

[1637] The server sends the generated user ID to the terminal.

[1638] 3. Initial Data Collection

[1639] The device collects initial data such as voice samples and heart rate to create an emotional profile of the user.

[1640] Input: User voice samples, heart rate, etc.

[1641] The device sends the collected data to a server to create an emotional profile.

[1642] Output: Emotion profile

[1643] Step 2: Route learning and configuration

[1644] 1. Set your destination

[1645] The user inputs a destination into the terminal.

[1646] Input: Destination information

[1647] The device acquires the current location information and sends it to the server.

[1648] Input: Current location information

[1649] Output: Current location information, destination information

[1650] 2. Route calculation

[1651] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past travel history, and specific requests.

[1652] Input: current location, destination, past travel history, specific requests

[1653] Output: Optimal route data

[1654] The server generates haptic feedback data based on the calculation results.

[1655] Output: Haptic feedback data

[1656] 3. Sending and storing feedback data

[1657] The server transmits the generated haptic feedback data to the terminal.

[1658] Input: Haptic feedback data

[1659] The device stores the received data and provides haptic feedback if necessary.

[1660] Output: Stored haptic feedback data

[1661] Step 3: Emotion recognition and adaptive feedback

[1662] 1. Sensor data collection

[1663] The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate in real time.

[1664] Input: Voice data, facial expression data, heart rate data

[1665] Output: Collected sensor data

[1666] 2. Emotional Recognition

[1667] The device's emotion engine analyzes the collected data and recognizes the user's emotions.

[1668] Input: Collected sensor data

[1669] Output: Emotional state data

[1670] The device sends the analysis results to the server.

[1671] 3. Adjusting feedback patterns

[1672] The server optimizes the haptic feedback pattern based on the user's emotional state.

[1673] Input: Emotional state data

[1674] Output: Tuned feedback pattern

[1675] The server retransmits the adjusted feedback pattern to the terminal.

[1676] The device applies the new feedback pattern and provides haptic feedback to the user.

[1677] (Application example 2)

[1678] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1679] In modern shopping experiences, users are often overwhelmed by a large amount of information, which can lead to feelings of stress and anxiety. However, there is a lack of technology that can recognize these emotions in real time and respond appropriately. In particular, virtual stores are required to provide appropriate feedback based on the user's emotional state, but no effective system exists for this purpose.

[1680] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1681] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and route conditions specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and route conditions and generating haptic feedback data, means for recognizing the user's emotions in real time and transmitting the data to the server, and means for receiving an optimal haptic feedback pattern from the server and providing it to the user. This makes it possible to reduce the stress and anxiety felt by the user during their shopping experience in a virtual store and provide appropriate haptic feedback and audio feedback.

[1682] "Current location information" is data indicating the location where the user is currently located.

[1683] "Destination information" is data indicating the location the user is trying to reach.

[1684] "Route conditions" are data indicating restrictions and requests regarding the route specified by the user.

[1685] "Haptic feedback data" is data that provides a physical response to the user's senses.

[1686] "Means for real-time emotion recognition" refers to a technical mechanism for instantly determining a user's current emotional state.

[1687] A "server" is a computer system that processes and stores data over a network.

[1688] "Feedback based on emotional state" refers to tactile, audio, or other responses provided in response to the user's emotions.

[1689] A "virtual store" is a virtual shopping environment created on the Internet.

[1690] "Haptic feedback" is a technology that conveys information to a user through physical sensation.

[1691] The present invention provides a system that recognizes a user's emotions in real time and provides adaptive haptic feedback. This system is comprised of a user terminal, a server, and a user, and has an emotion engine built in.

[1692] System Configuration

[1693] The user terminal (smart glasses or smartphone) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time to generate optimal haptic feedback.

[1694] User registration and initial settings

[1695] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[1696] Route learning and configuration

[1697] When a user starts shopping in a virtual store, the device sends its current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past behavior history, and specific requests. It generates haptic feedback data and sends it to the device. The device stores this data and provides haptic feedback when needed.

[1698] Emotion Recognition and Adaptive Feedback

[1699] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or anxious, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[1700] Specific examples

[1701] A user wears smart glasses and visits a virtual store. While searching for a product in the store, the emotion engine detects the user's anxiety and sends the emotion data to the server. The server calculates the optimal haptic feedback pattern and sends it to the device. The device then provides vibration and audio feedback, reducing the user's stress.

[1702] Prompt Sentence Examples

[1703] It recognizes the user's emotions while selecting products in a virtual store and provides immediate relaxation feedback.

[1704] It detects the stress felt by users searching for a specific product in a virtual store and provides haptic feedback.

[1705] This system and method provide optimal haptic feedback based on the user's emotional state, enabling a comfortable shopping experience in a virtual store.

[1706] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1707] Step 1:

[1708] The user turns on the device and enters basic information (name, age, specific requests).

[1709] Specific operation: The user enters their name, age, and specific requests into the terminal, and then sends that information to the server by pressing the send button.

[1710] Input: User's basic information (name, age, specific needs).

[1711] Data processing / calculation: The basic information entered by the terminal is encrypted and sent to the server.

[1712] Output: Encrypted user data.

[1713] Step 2:

[1714] The server stores the received user information in a database, generates a user ID, and sends it to the terminal.

[1715] Specific operation: The server saves the received user information in a database, generates a new user ID, and returns the ID to the device.

[1716] Input: Encrypted user data.

[1717] Data processing / calculation: Saving user data and generating user IDs.

[1718] Output: The generated user ID.

[1719] Step 3:

[1720] The device collects initial data to generate an emotional profile of the user.

[1721] Specific operation: The device records initial emotion data using the camera, microphone, heart rate sensor, etc.

[1722] Input: User's voice, facial expressions, heart rate, etc.

[1723] Data processing / computation: Collecting and recording emotional data.

[1724] Output: Initial emotion profile data.

[1725] Step 4:

[1726] When a user starts shopping in a virtual store, the terminal transmits current location information to the server.

[1727] Specific operation: The device uses the GPS module to obtain its current location information and sends it to the server.

[1728] Input: The user's current location.

[1729] Data processing / calculation: Acquisition and transmission of location information.

[1730] Output: Current location information sent to the server.

[1731] Step 5:

[1732] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past behavior, and specific requests.

[1733] Specific operation: Based on the data received by the server, the generative AI model is used to calculate the optimal shopping route.

[1734] Input: current location, destination, past activity history, specific requests.

[1735] Data processing / calculation: Route calculation using generative AI models.

[1736] Output: Optimal route data.

[1737] Step 6:

[1738] Haptic feedback data is generated based on the optimal path and transmitted to the device.

[1739] Specific operation: The server generates haptic feedback data based on the route data calculated and sends it to the device.

[1740] Input: Optimal route data.

[1741] Data processing / computation: Generation of haptic feedback data.

[1742] Output: Haptic feedback data.

[1743] Step 7:

[1744] The terminal stores the received haptic feedback data and provides haptic feedback when necessary.

[1745] Specific operation: The device stores the received data in its internal storage and provides it to the user via the haptic feedback device when necessary.

[1746] Input: Haptic feedback data.

[1747] Data processing / calculation: Storing data and giving instructions to the haptic device.

[1748] Output: Haptic feedback.

[1749] Step 8:

[1750] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate.

[1751] Specific operation: The device inputs data acquired by sensors into the emotion engine and analyzes the user's emotions in real time.

[1752] Input: speech, facial expression, and heart rate data.

[1753] Data processing / computation: Analysis of emotion data.

[1754] Output: User's emotional state data.

[1755] Step 9:

[1756] If the user is feeling stressed or anxious, the device sends that information to the server.

[1757] Specific operation: The terminal transmits the user's emotional state to the server, and the server receives it.

[1758] Input: Emotional state data.

[1759] Data processing / calculation: Sending and receiving emotional data.

[1760] Output: Emotion data sent to the server.

[1761] Step 10:

[1762] The server calculates the optimal haptic feedback pattern and sends it to the device.

[1763] Specific operation: Based on the emotional state, the server adjusts the haptic feedback, generates new patterns, and sends them to the device.

[1764] Input: Emotional state data.

[1765] Data processing / calculation: Generation of optimal haptic feedback patterns.

[1766] Output: New haptic feedback data.

[1767] Step 11:

[1768] The device provides new haptic feedback to the user, reducing stress and anxiety.

[1769] Specific operation: The device activates the haptic feedback device based on the new haptic feedback data, providing the user with a sense of security.

[1770] Input: New haptic feedback data.

[1771] Data processing / computation: Implementing haptic feedback.

[1772] Output: Improved user emotional state.

[1773] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1774] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1775] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1776] [Fourth embodiment]

[1777] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1778] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1779] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1780] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1781] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1782] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1783] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1784] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1785] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1786] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1787] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1788] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1789] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1790] The present invention provides a system for safely guiding a user to a destination using haptic feedback. This allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own. An embodiment of the present invention will be described below.

[1791] System Configuration

[1792] This system consists of a user terminal (a smartphone or dedicated device), a server connected to it, and the user. Communication between each component is carried out via a network (e.g., the Internet).

[1793] Program processing

[1794] User registration and initial settings

[1795] When a user uses the system for the first time, they start up their device, log in, and enter basic information (name, age, specific requests). The device encrypts this information and sends it to the server. The server stores the received information in a database, generates a unique user identification number, and sends it back to the device. This makes it possible to manage the user's individual data and movement history.

[1796] Route learning and configuration

[1797] When a user heads to a specific destination, they input destination information from their device. The device acquires GPS data and sends it to the server along with their current location. Based on this information, the server uses a generative AI model to calculate the optimal route. This optimal route takes into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes). The calculated route is generated as haptic feedback data and sent to the device. The device stores this haptic feedback data and uses it when necessary.

[1798] Specific examples of real-time guidance

[1799] When Elderly Person A goes to the hospital

[1800] Elderly user A holds the device and requests route guidance to the hospital. The device acquires its current location information and sends it to the server with the hospital as the destination. The server generates an optimal route based on past movement patterns (e.g., a route with few slopes) and sends this as haptic feedback data to the device. The device then uses the haptic data to guide Elderly User A using a vibration pattern. If Elderly User A goes in the wrong direction, the device will warn them with a different vibration pattern and show them the correct direction. Finally, the server confirms that Elderly User A has reached the hospital and notifies them of the completion using haptic feedback.

[1801] If your child goes to school

[1802] A child's parent pre-registers their child's route to school on the device. As the child carries the device to school, the device tracks their progress in real time and provides guidance. The server periodically receives location information and checks whether the child is progressing correctly. If the child takes a wrong turn, the server calculates a corrected route and sends new haptic feedback data. The device then provides haptic feedback accordingly, correcting the route. When the child arrives at school, a notification is sent to the parent, providing peace of mind.

[1803] The above is an embodiment of the present invention, which is a system that helps various users reach their destination safely and easily. This system provides route guidance through tactile feedback without relying on vision, so anyone can easily use it.

[1804] The processing flow will be explained below.

[1805] Program processing details

[1806] User registration and initial settings

[1807] Step 1:

[1808] The user starts up the device and enters basic information such as their name, age, email address, and password into the application's login screen.

[1809] Step 2:

[1810] The terminal encrypts the basic information entered and transmits it to the server via the network.

[1811] Step 3:

[1812] The server stores the received basic information in the database and generates a new user ID.

[1813] Step 4:

[1814] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[1815] Step 5:

[1816] The device saves the received user ID and completes the initial setup.

[1817] Route learning and configuration

[1818] Step 1:

[1819] The user operates the terminal to select a destination and start setting route guidance.

[1820] Step 2:

[1821] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[1822] Step 3:

[1823] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[1824] Step 4:

[1825] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[1826] Step 5:

[1827] The terminal stores the received haptic feedback data and provides it to the user as needed.

[1828] Real-time guidance

[1829] Step 1:

[1830] To start guidance along the route specified by the user, route guidance is started on the terminal.

[1831] Step 2:

[1832] The device periodically acquires GPS data and tracks its current location.

[1833] Step 3:

[1834] The device sends its current location information to the server.

[1835] Step 4:

[1836] The server evaluates the progress based on the current position, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[1837] Step 5:

[1838] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path and generates new haptic feedback data to send to the device.

[1839] Step 6:

[1840] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern. If the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[1841] Step 7:

[1842] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[1843] Specific examples

[1844] When Elderly Person A goes to the hospital

[1845] Step 1:

[1846] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[1847] Step 2:

[1848] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[1849] Step 3:

[1850] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[1851] Step 4:

[1852] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[1853] Step 5:

[1854] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[1855] Step 6:

[1856] When Elderly A reaches his destination, the hospital, the server confirms his arrival and sends a completion notification to the terminal, which then notifies Elderly A via haptic feedback.

[1857] In this way, the system can effectively guide the user to their destination using haptic feedback.

[1858] Example 1

[1859] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1860] To provide a system that enables the visually impaired, the elderly, children, etc. to safely reach their destinations on their own. In particular, to improve the safety and convenience of users by enabling them to receive route guidance without relying on visual information and by providing appropriate guidance even when heading in the wrong direction.

[1861] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1862] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and travel conditions specified by the user, means for calculating an optimal route using a generative AI model based on the acquired current location information, destination information, and travel conditions, and means for generating haptic feedback data, and means for generating new haptic feedback data. This enables anyone to easily and safely reach their destination without relying on vision.

[1863] A "user terminal" is an electronic device carried by users such as visually impaired people, the elderly, and children, and is primarily used to obtain GPS data and provide tactile feedback.

[1864] The "server" is a central computer system that receives data sent from user terminals, calculates optimal routes, generates haptic feedback data, and stores the user's movement history.

[1865] "Current location information" is real-time location data of a user obtained using the GPS function of the user terminal.

[1866] "Destination information" refers to geographical information of a place that a user wants to reach, which is input into the terminal.

[1867] "Travel conditions" is information including specific requests and constraints specified by the user (e.g., avoiding slopes, width of sidewalks, etc.).

[1868] A "generative AI model" is an artificial intelligence technology used by the server, and is an algorithm that calculates the optimal route based on past data and user characteristics.

[1869] "Haptic feedback data" is data that is sent to a user terminal and is used to provide route guidance to the user in the form of vibration patterns or the like.

[1870] A "corrected route" is a new optimal route that is recalculated when the user takes a wrong turn.

[1871] "Arrival confirmation" is the process by which the server confirms that the user has reached their destination.

[1872] "Haptic feedback" is a means of physically conveying route guidance and warnings to users using the device's vibration function, etc.

[1873] System Overview

[1874] The present invention is a system for safely guiding a user to a destination using haptic feedback, which allows the visually impaired, the elderly, children, and others to easily reach their destinations on their own.

[1875] System Configuration

[1876] The system consists of the following components:

[1877] 1. User device (e.g., smartphone or dedicated device)

[1878] 2. Server

[1879] 3. Users

[1880] Communication takes place over the Internet.

[1881] Program processing

[1882] User registration and initial settings

[1883] When a user starts up the device for the first time, they are taken to a login screen where they enter basic information such as their name, age, and any specific requests.

[1884] The terminal uses AES (Advanced Encryption Standard) to encrypt the basic information entered and sends this encrypted data to the server.

[1885] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[1886] Route learning and configuration

[1887] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[1888] The device uses a GPS module to obtain data on its current location and transmits this information to a server.

[1889] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests (e.g., avoiding slopes).

[1890] The calculated path is generated as haptic feedback data, which is sent to the device, which stores the haptic feedback data in its internal memory and uses it as needed.

[1891] Real-time guidance

[1892] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[1893] The server periodically receives the user's location information and verifies that the route is correct.

[1894] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[1895] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[1896] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data to the device as a completion notification. This feedback data conveys the end notification to the user.

[1897] Specific examples

[1898] When Elderly Person A goes to the hospital

[1899] 1. The user, elderly person A, takes the device and requests route guidance to the hospital.

[1900] 2. The device obtains its current location information using the GPS module and sends it to the server with the hospital as the destination.

[1901] 3. The server generates the optimal route based on past travel patterns and basic information.

[1902] 4. The device guides Elderly A using vibration patterns based on the haptic feedback data. For example, a right turn is indicated by three short vibrations, and a left turn is indicated by one long vibration.

[1903] 5. If Elderly Person A goes in the wrong direction, the device will warn him with five short vibrations and provide him with a recalculated correct route.

[1904] 6. The server confirms that the delivery has reached the hospital and notifies the completion with a "3-second continuous vibration."

[1905] If your child goes to school

[1906] 1. Parents register their child's route to school on the device in advance.

[1907] 2. When a child carries a device and heads to school, the device tracks their location in real time.

[1908] 3. Periodically send location information to the server to confirm correct progress.

[1909] 4. If the child takes a wrong turn, the server instantly calculates a corrective path and sends a new vibration pattern to the device.

[1910] 5. When the child reaches the destination (school), an "arrival notification" is sent to the parent, providing peace of mind to the parent.

[1911] Examples of prompt statements

[1912] "Generate route guidance for an elderly person going to the hospital based on the following information: Name - Ichiro Suzuki, Age - 70, Current location - {Current location coordinates}, Destination - {Hospital coordinates}, Past travel patterns - {Past travel history}"

[1913] This provides a system that helps users reach their destination safely and easily.

[1914] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1915] Program processing steps

[1916] User registration and initial settings

[1917] Step 1:

[1918] A user starts up the device for the first time and is directed to a login screen, where they enter basic information such as their name, age, and any specific requests.

[1919] Input: Name, age, specific requests

[1920] Output: Basic information data

[1921] Step 2:

[1922] The terminal acquires the basic information entered and encrypts it using AES (Advanced Encryption Standard).

[1923] Input: Basic information data

[1924] Output: Encrypted basic information data

[1925] Step 3:

[1926] The terminal transmits the encrypted basic information data to the server.

[1927] Input: Encrypted basic information data

[1928] Output: None (data transmission completed)

[1929] Step 4:

[1930] The server decrypts the received encrypted data and stores it in a database (e.g., MySQL). At the same time, it generates a unique user identification number and sends it back to the terminal.

[1931] Input: Encrypted basic information data

[1932] Output: A unique identification number for the user

[1933] Route learning and configuration

[1934] Step 5:

[1935] If the user wishes to set a specific destination, the user inputs the destination information using an application on the terminal.

[1936] Input: Destination information

[1937] Output: None (Destination information input complete)

[1938] Step 6:

[1939] The device uses a GPS module to obtain data on its current location and sends this information along with basic and destination information to a server.

[1940] Input: GPS data, basic information, destination information

[1941] Output: None (data transmission completed)

[1942] Step 7:

[1943] The server uses a generative AI model (e.g., TensorFlow) to calculate the optimal route, taking into account the user's basic information, past travel history, and specific requests.

[1944] Input: GPS data, basic information, destination information, past travel history, specific requests

[1945] Output: Optimal route data

[1946] Step 8:

[1947] The server generates the calculated optimal route data as haptic feedback data and transmits this data to the terminal.

[1948] Input: Optimal route data

[1949] Output: Haptic feedback data

[1950] Step 9:

[1951] The device stores the received haptic feedback data in its internal memory and uses it as needed.

[1952] Input: Haptic feedback data

[1953] Output: None (data saving completed)

[1954] Real-time guidance

[1955] Step 10:

[1956] When a user heads to a destination, the device's GPS function is used to obtain the user's current location in real time.

[1957] Input: None (real-time location acquisition)

[1958] Output: Current location information

[1959] Step 11:

[1960] The server periodically receives the user's location information and verifies that the route is correct.

[1961] Input: Current location information

[1962] Output: Correct route confirmation result

[1963] Step 12:

[1964] The device uses haptic feedback to guide the user through the route with appropriate vibration patterns, such as repeated short vibrations if a right turn is required and longer vibrations if a left turn is required.

[1965] Input: haptic feedback data, current location information

[1966] Output: Vibration pattern

[1967] Step 13:

[1968] If the user makes a wrong turn, the server instantly calculates a corrective path and generates new haptic feedback data, which is then sent to the device, which responds by providing a different vibration pattern to correct the path.

[1969] Input: current location, inaccurate heading

[1970] Output: New haptic feedback data, vibration patterns

[1971] Step 14:

[1972] When the user reaches the destination, the server confirms the arrival and sends haptic feedback data as a completion notification to the device, which then receives the data and notifies the user.

[1973] Input: Confirmation of arrival at destination

[1974] Output: Haptic feedback data and vibration pattern for completion notification

[1975] This allows the user to reach their destination safely and easily.

[1976] (Application example 1)

[1977] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1978] People with visual impairments, the elderly, and children face challenges in finding the products they are looking for in physical stores. Navigating safely and efficiently without relying on vision is difficult. Furthermore, there is a lack of systems that provide appropriate feedback to users when they have taken a wrong turn or when they have reached the desired product. This makes in-store shopping stressful and inconvenient for many users.

[1979] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1980] In this invention, the server includes means for acquiring user's current location information, means for acquiring user-specified target product information and in-store conditions, and means for calculating an optimal route based on the acquired current location information, target product information, and in-store conditions, and generating haptic feedback data, thereby enabling visually impaired people, the elderly, and children to move safely and efficiently within a physical store and reach their target products.

[1981] "Current location information" is data for identifying the user's current location.

[1982] "Target product information" is information about a specific product or product category that the user wants to find in the store.

[1983] "In-store conditions" refers to information about the structure and layout of the store, and conditions for route selection (e.g., passable aisles, product display positions).

[1984] "Calculate route" is a process of calculating the optimal route for the user to reach the target product based on current location information, target product information, and in-store conditions.

[1985] "Haptic feedback data" is data that a device uses to communicate information to a user through the sense of touch, specifically including sensory signals such as vibration and pressure.

[1986] A "correction route" is a new path that the user can take to get back on the correct route if they take a wrong turn.

[1987] The "arrival notification" is a notification accompanied by haptic feedback to inform the user that they have reached the desired product.

[1988] To realize this invention, a system that links the functions of a user terminal (such as a smartphone), a server, and the user is required. This system uses haptic feedback to help users find the product they are looking for in a physical store.

[1989] System Configuration

[1990] User terminal

[1991] The user terminal is a device that the user can carry around, such as a smartphone or a dedicated device. It includes location information acquisition functions such as GPS or beacons (Bluetooth) and vibration motors to provide haptic feedback. The user terminal communicates with the server via the Internet or Bluetooth.

[1992] server

[1993] The server uses cloud services such as Amazon Web Services (AWS). Platforms such as AWS EC2 and AWS Lambda are used to execute the series of processes. MySQL is also used as the database. The server receives the user's location information and desired product information, and uses a generative AI model (such as OpenAI's GPT-3) to calculate the optimal route.

[1994] Program processing

[1995] 1. User registration and initial settings

[1996] A user installs and launches the app for the first time, and enters their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[1997] This information is encrypted by the device and sent to an AWS server, which stores it in a database and generates a unique identification number for the user.

[1998] 2. In-store location and routing

[1999] When a user enters a store, the device obtains its current location using GPS or a beacon system.

[2000] Users use the app to input the desired product name or category, such as "Milk section."

[2001] The server receives this information and uses a generative AI model to calculate the optimal path and generate haptic feedback data.

[2002] 3. Real-time route guidance

[2003] Based on the haptic feedback data sent from the server, the device uses vibration patterns to provide guidance to the user.

[2004] If the user takes a wrong turn, the device calculates a corrective path and provides new haptic feedback data.

[2005] When the user arrives at the desired product, they are notified of the arrival via haptic feedback.

[2006] Specific examples

[2007] For example, if a user selects the "Milk Corner" as their destination, the app retrieves their current location and calculates the optimal route. Using a generative AI model, the app presents prompts that suggest the clearest and safest route for the user. If the user takes a wrong turn, the server calculates a new route and the device corrects the guidance with haptic feedback.

[2008] Example prompt sentence:

[2009] Japanese

[2010] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[2011] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2012] Step 1:

[2013] User registration and initial settings

[2014] Users install and launch the app, then enter their name, age, and any special needs (such as visual impairment or needing to avoid certain roads) on the login screen.

[2015] The terminal encrypts this input information and transmits it to the server.

[2016] The server stores the received information in a MySQL database, generates a unique identification number for the user, and sends it back to the terminal.

[2017] Input: User information (name, age, needs).

[2018] Output: A unique identification number for the user.

[2019] Step 2:

[2020] Obtaining location information within the store

[2021] A user enters a store.

[2022] The device obtains its current location information using GPS or a beacon system (Bluetooth).

[2023] Once the current location information is obtained, it is sent to the server.

[2024] Input: None.

[2025] Output: Current location of the user.

[2026] Step 3:

[2027] Enter the desired product information

[2028] Users enter the desired product name or category into the app, such as "Milk section."

[2029] The terminal transmits the input target product information to the server.

[2030] Input: Desired product information.

[2031] Output: None.

[2032] Step 4:

[2033] Optimal Route Calculation

[2034] Based on the received current location information and desired product information, the server calculates the optimal route using a generative AI model (e.g., OpenAI GPT-3) taking into account in-store conditions.

[2035] The server generates the calculated optimal path as haptic feedback data and transmits it to the terminal.

[2036] Input: current location information, desired product information, in-store conditions.

[2037] Output: Haptic feedback data.

[2038] Step 5:

[2039] Providing haptic feedback

[2040] The device generates a vibration pattern based on the received haptic feedback data and provides guidance to the user through haptic feedback.

[2041] Input: Haptic feedback data.

[2042] Output: Haptic feedback via vibration.

[2043] Step 6:

[2044] Check and correct heading

[2045] If the user takes a wrong turn, the device will detect the heading error based on data from beacons or GPS.

[2046] If the terminal detects an error, it sends the information to the server.

[2047] The server calculates the corrective path and generates new haptic feedback data to send to the terminal.

[2048] Input: Heading error information.

[2049] Output: New haptic feedback data.

[2050] Step 7:

[2051] Confirmation that the desired product has arrived

[2052] When the user reaches the desired product, the terminal confirms arrival based on the location information.

[2053] The server generates data to notify the device of the arrival using haptic feedback and sends it to the device.

[2054] Input: Arrival location information.

[2055] Output: Reach haptic feedback data.

[2056] For example, here is the prompt again:

[2057] Japanese

[2058] The user selected the "Milk Corner" as their destination. The app calculated the optimal route based on their current location and generated haptic feedback data. If the user took a wrong turn, the app calculated a new route and corrected the guidance with an appropriate vibration pattern.

[2059] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2060] The present invention is a system that recognizes a user's emotions and provides adaptive haptic feedback. This system is composed of a user terminal, a server, and a user, and has an emotion engine built in.

[2061] System Configuration

[2062] The user terminal (smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[2063] User registration and initial settings

[2064] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[2065] Route learning and configuration

[2066] The user inputs their destination, and the device sends their current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past travel history, and specific requests. Haptic feedback data is generated and sent to the device. The device stores this data and provides haptic feedback when needed.

[2067] Emotion Recognition and Adaptive Feedback

[2068] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or tense, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[2069] Specific examples

[2070] When Elderly Person A goes to the hospital

[2071] Elderly user A starts the device and requests route guidance to the hospital. The device acquires current location information and sends destination information to the server. The server generates an optimal route and sends haptic feedback data to the device. The device provides appropriate haptic feedback to Elderly User A. If Elderly User A goes in the wrong direction, the device provides warning haptic feedback and suggests the correct direction.

[2072] If the device detects that Elderly A is feeling stressed or tense during the interaction, the device's emotion engine sends that information to the server, which can then adjust the feedback pattern and resend the haptic feedback to provide a sense of security.

[2073] If your child goes to school

[2074] A parent sets their child's school route on the device. As the child heads to school, the device tracks their progress in real time and provides route guidance. The server periodically receives location information to check whether the user is heading in the correct direction. If the user heads in the wrong direction, the server calculates a corrected route and sends new haptic feedback data to the device, which then provides a warning.

[2075] If the child feels tired during the process, the emotion engine sends that information to the server, which then provides haptic feedback to encourage them to take a break.

[2076] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[2077] The processing flow will be explained below.

[2078] Program processing details

[2079] User registration and initial settings

[2080] Step 1:

[2081] The user starts up the device and enters basic information such as their name, age, email address, and password on the login screen.

[2082] Step 2:

[2083] The terminal encrypts the basic information entered and transmits it to the server via the network.

[2084] Step 3:

[2085] The server stores the received basic information in the database and generates a new user ID.

[2086] Step 4:

[2087] A notification message of successful registration including the server-generated user ID is sent to the terminal.

[2088] Step 5:

[2089] The device stores the received user ID and initializes the emotion engine to collect initial data to generate the user's emotion profile.

[2090] Route learning and configuration

[2091] Step 1:

[2092] The user operates the terminal to select a destination and start setting route guidance.

[2093] Step 2:

[2094] The device uses GPS data to obtain the user's current location and sends it to the server along with destination information.

[2095] Step 3:

[2096] The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, and the user's specific requests (e.g., avoiding slopes).

[2097] Step 4:

[2098] The server converts the calculated optimal path into haptic feedback data and transmits the data to the terminal.

[2099] Step 5:

[2100] The terminal stores the received haptic feedback data and provides it to the user as needed.

[2101] Emotion Recognition and Adaptive Feedback

[2102] Step 1:

[2103] Guidance will begin along the route specified by the user on the terminal.

[2104] Step 2:

[2105] The device periodically acquires GPS data and tracks its current location.

[2106] Step 3:

[2107] The device recognizes the user's emotions using sensor data such as voice, facial expressions, and heart rate.

[2108] Step 4:

[2109] The device transmits the recognized emotion data to the server.

[2110] Step 5:

[2111] The server evaluates the progress and emotion data based on the current location, and if the direction of progress is correct, creates the next haptic feedback data and sends it to the terminal.

[2112] Step 6:

[2113] If the server determines that the user's direction of travel is incorrect, it calculates a corrective path, generates new haptic feedback data, and sends it to the terminal.

[2114] Step 7:

[2115] Based on the haptic feedback data received by the device, the device guides the user with an appropriate vibration pattern: if the direction is correct, the device continues the guidance, and if the direction is wrong, the device provides a warning vibration.

[2116] Step 8:

[2117] The device adjusts the feedback pattern according to the user's emotions, providing haptic feedback to relieve stress and tension, while simultaneously providing haptic feedback to encourage rest if the user's emotions indicate fatigue.

[2118] Step 9:

[2119] The server confirms that the user has reached the destination and sends a notification of arrival to the terminal, which then notifies the user with haptic feedback.

[2120] Specific examples

[2121] When Elderly Person A goes to the hospital

[2122] Step 1:

[2123] The user, elderly person A, starts up the terminal and requests route guidance to the hospital.

[2124] Step 2:

[2125] The terminal acquires the current location information and transmits the destination information "hospital" to the server.

[2126] Step 3:

[2127] The server generates an optimal route based on past travel history and specific requests (e.g., a route with fewer slopes) and sends haptic feedback data to the device.

[2128] Step 4:

[2129] Based on the tactile data received by the device, guidance is given to elderly person A using vibration patterns.

[2130] Step 5:

[2131] If Elderly A goes in the wrong direction, the device will warn him with a different vibration pattern and point him in the right direction.

[2132] Step 6:

[2133] The device recognizes the emotions of elderly person A using voice, facial expression, and heart rate sensors, and transmits the emotional data to the server.

[2134] Step 7:

[2135] The server adjusts the feedback pattern based on the received emotional data, regenerates haptic feedback that provides a sense of security, and sends it to the device.

[2136] Step 8:

[2137] When Elderly A reaches the hospital, the server confirms the arrival and sends a completion notification to the terminal, which then notifies Elderly A of the arrival via haptic feedback.

[2138] If your child goes to school

[2139] Step 1:

[2140] Parents can register their children's routes to school in advance on their devices.

[2141] Step 2:

[2142] As a child walks to school, the device tracks their progress in real time and provides route guidance.

[2143] Step 3:

[2144] The server periodically receives location information and checks whether the user is heading in the correct direction.

[2145] Step 4:

[2146] If a wrong turn occurs, the server calculates a corrective path and sends new haptic feedback data to the device.

[2147] Step 5:

[2148] The device will guide you in the right direction based on the new haptic feedback it receives, and adjust the vibration pattern to a more reassuring one if the user is feeling stressed or tense.

[2149] Step 6:

[2150] If the device senses fatigue, it will provide haptic feedback to encourage you to take a break.

[2151] Step 7:

[2152] When the child arrives at school, the server confirms the arrival, sends a notification of arrival to the terminal, and notifies the parents.

[2153] In this way, the system can recognize the user's emotions and provide more personalized route guidance through haptic feedback, helping the user reach their destination with confidence.

[2154] Example 2

[2155] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2156] Conventional route guidance systems can obtain a user's current location information and provide the optimal route to a destination, but they do not provide feedback that takes into account the user's emotional state. In particular, there is a lack of means to help users with special needs, such as the elderly and children, safely reach their destination while reducing stress and tension. Furthermore, there is a need for a system that is flexible enough to provide appropriate real-time feedback regarding incorrect directions.

[2157] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and requirements specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and requirements and generating haptic feedback data using a generative AI model, means for providing the generated haptic feedback data to the user together with haptic feedback, means for collecting sensor data and recognizing the user's emotions in real time, and means for adjusting the haptic feedback pattern according to the user's emotions. This not only enables the user to reach their destination safely, but also makes it possible to provide appropriate feedback in real time according to errors in heading or the user's emotional state.

[2158] "Means for obtaining the user's current location information" refers to a function for identifying and obtaining the user's current geographical location using a GPS module or other location information services from the device the user is using.

[2159] The "means for obtaining user-specified destination information and requirements" refers to a function for collecting the specific destination and the user's specific wishes and requests that the user has input into the system.

[2160] "Means for calculating the optimal route and generating haptic feedback data using a generative AI model" refers to a function that uses an AI model to calculate the optimal route for the user based on the acquired current location information, destination information, and requirements, and creates data for haptic feedback.

[2161] The "means for providing the generated haptic feedback data to the user along with haptic feedback" is a function for using the generated haptic feedback data to provide a haptic notification to the user's device.

[2162] "Means for collecting sensor data and recognizing user emotions in real time" refers to a function that uses sensors installed in the user's device to collect and analyze data such as voice, facial expressions, and heart rate, in order to recognize the user's emotional state in real time.

[2163] The "means for adjusting the haptic feedback pattern in accordance with the user's emotion" is a function for appropriately changing the intensity and pattern of the haptic feedback based on the recognized emotion of the user.

[2164] MODE FOR CARRYING OUT THE INVENTION

[2165] The present invention provides a system for recognizing a user's emotions and adaptively providing haptic feedback. The system is comprised of a user terminal, a server, and a user, and has an emotion engine built in. Specific embodiments of the system are described below.

[2166] System Configuration

[2167] The user terminal (e.g., a smartphone or dedicated device) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time and generates optimal haptic feedback.

[2168] Specific functions and operations

[2169] User registration and initial settings

[2170] 1. The user starts the device and enters basic information (name, age, specific requests).

[2171] 2. The device encrypts the entered information and sends it to the server.

[2172] 3. The server stores the information in a database, generates a user ID, and sends it to the device.

[2173] 4. The device collects initial data such as voice samples and heart rate to create an emotional profile.

[2174] Route learning and configuration

[2175] 1. The user enters the destination, and the device obtains the current location information and sends it to the server.

[2176] 2. The server uses a generative AI model to calculate the optimal route based on the current location, destination, past travel history, and specific requests.

[2177] 3. The server generates haptic feedback data and sends it to the device.

[2178] 4. The device stores the received data and provides haptic feedback when necessary.

[2179] Emotion Recognition and Adaptive Feedback

[2180] 1. The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate.

[2181] 2. The device analyzes the data collected and recognizes the user's emotions.

[2182] 3. The device sends the recognition results to the server.

[2183] 4. The server adjusts the haptic feedback pattern according to the user's emotional state and sends it back to the device.

[2184] 5. The device applies the new feedback pattern and provides haptic feedback to the user.

[2185] Specific examples

[2186] Example 1: Elderly person A goes to the hospital

[2187] 1. The user (elderly person A) starts up the device and requests "route guidance to the hospital."

[2188] 2. The device obtains the current location information using the GPS module and sends the destination information (hospital) to the server.

[2189] 3. The server generates the optimal path and sends haptic feedback data to the device.

[2190] 4. The device displays route directions and provides haptic feedback.

[2191] 5. If Elderly Person A goes in the wrong direction, the device will generate haptic feedback to warn him and show him the correct direction to go.

[2192] 6. If Elderly Person A feels stressed during the exercise, the device's emotion engine detects an increase in heart rate and sends the emotion data to the server.

[2193] 7. The server calculates reassuring haptic feedback and sends it to the device.

[2194] 8. The device provides new haptic feedback to reassure Elderly A.

[2195] Example 2: When a child goes to school

[2196] 1. Parents set their child's school route on the device.

[2197] 2. When the child starts going to school, the device tracks their current location and sends it to the server.

[2198] 3. The server periodically checks the location information and monitors the direction of travel.

[2199] 4. If the child takes a wrong turn, the server calculates a corrective path and sends new haptic feedback data to the device.

[2200] 5. The device will provide warnings and give you precise directions to follow.

[2201] 6. The device's emotion engine detects the child's fatigue and sends the data to the server.

[2202] 7. The server provides haptic feedback to encourage a break, and the device notifies the child.

[2203] Prompt Sentence Examples

[2204] Route guidance request

[2205] The user types into the terminal, "I want to go to the hospital. Please tell me the best route."

[2206] Emotional state report

[2207] The user types into the device, "I'm feeling a little stressed right now. Please provide some haptic feedback to help me relax."

[2208] As described above, this system recognizes the user's emotions in real time and provides optimal haptic feedback, allowing the user to reach their destination with peace of mind. Furthermore, by providing appropriate feedback in real time depending on the user's emotional state and direction, the system offers a safer and more comfortable travel experience.

[2209] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2210] Step 1: User registration and initial setup

[2211] 1. Start the device and enter basic information

[2212] The user turns on the device and enters basic information such as name, age and specific requests.

[2213] Input: Basic information (name, age, specific requests)

[2214] The terminal encrypts the information entered.

[2215] Output: Encrypted basic information

[2216] 2. Transmission and storage of information

[2217] The device sends the encrypted information to the server.

[2218] Input: Encrypted basic information

[2219] The server stores the received information in a database and generates a user ID.

[2220] Output: User ID

[2221] The server sends the generated user ID to the terminal.

[2222] 3. Initial Data Collection

[2223] The device collects initial data such as voice samples and heart rate to create an emotional profile of the user.

[2224] Input: User voice samples, heart rate, etc.

[2225] The device sends the collected data to a server to create an emotional profile.

[2226] Output: Emotion profile

[2227] Step 2: Route learning and configuration

[2228] 1. Set your destination

[2229] The user inputs a destination into the terminal.

[2230] Input: Destination information

[2231] The device acquires the current location information and sends it to the server.

[2232] Input: Current location information

[2233] Output: Current location information, destination information

[2234] 2. Route calculation

[2235] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past travel history, and specific requests.

[2236] Input: current location, destination, past travel history, specific requests

[2237] Output: Optimal route data

[2238] The server generates haptic feedback data based on the calculation results.

[2239] Output: Haptic feedback data

[2240] 3. Sending and storing feedback data

[2241] The server transmits the generated haptic feedback data to the terminal.

[2242] Input: Haptic feedback data

[2243] The device stores the received data and provides haptic feedback if necessary.

[2244] Output: Stored haptic feedback data

[2245] Step 3: Emotion recognition and adaptive feedback

[2246] 1. Sensor data collection

[2247] The device's emotion engine collects sensor data such as the user's voice, facial expressions, and heart rate in real time.

[2248] Input: Voice data, facial expression data, heart rate data

[2249] Output: Collected sensor data

[2250] 2. Emotional Recognition

[2251] The device's emotion engine analyzes the collected data and recognizes the user's emotions.

[2252] Input: Collected sensor data

[2253] Output: Emotional state data

[2254] The device sends the analysis results to the server.

[2255] 3. Adjusting feedback patterns

[2256] The server optimizes the haptic feedback pattern based on the user's emotional state.

[2257] Input: Emotional state data

[2258] Output: Tuned feedback pattern

[2259] The server retransmits the adjusted feedback pattern to the terminal.

[2260] The device applies the new feedback pattern and provides haptic feedback to the user.

[2261] (Application example 2)

[2262] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2263] In modern shopping experiences, users are often overwhelmed by a large amount of information, which can lead to feelings of stress and anxiety. However, there is a lack of technology that can recognize these emotions in real time and respond appropriately. In particular, virtual stores are required to provide appropriate feedback based on the user's emotional state, but no effective system exists for this purpose.

[2264] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2265] In this invention, the server includes means for acquiring current location information of the user, means for acquiring destination information and route conditions specified by the user, means for calculating an optimal route based on the acquired current location information, destination information, and route conditions and generating haptic feedback data, means for recognizing the user's emotions in real time and transmitting the data to the server, and means for receiving an optimal haptic feedback pattern from the server and providing it to the user. This makes it possible to reduce the stress and anxiety felt by the user during their shopping experience in a virtual store and provide appropriate haptic feedback and audio feedback.

[2266] "Current location information" is data indicating the location where the user is currently located.

[2267] "Destination information" is data indicating the location the user is trying to reach.

[2268] "Route conditions" are data indicating restrictions and requests regarding the route specified by the user.

[2269] "Haptic feedback data" is data that provides a physical response to the user's senses.

[2270] "Means for real-time emotion recognition" refers to a technical mechanism for instantly determining a user's current emotional state.

[2271] A "server" is a computer system that processes and stores data over a network.

[2272] "Feedback based on emotional state" refers to tactile, audio, or other responses provided in response to the user's emotions.

[2273] A "virtual store" is a virtual shopping environment created on the Internet.

[2274] "Haptic feedback" is a technology that conveys information to a user through physical sensation.

[2275] The present invention provides a system that recognizes a user's emotions in real time and provides adaptive haptic feedback. This system is comprised of a user terminal, a server, and a user, and has an emotion engine built in.

[2276] System Configuration

[2277] The user terminal (smart glasses or smartphone) includes a GPS module, a haptic feedback device, and an emotion engine. The server processes the location information and emotion data in real time to generate optimal haptic feedback.

[2278] User registration and initial settings

[2279] The user turns on the device and enters basic information (name, age, specific needs). The device encrypts this information and sends it to the server, which stores it in a database, generates a user ID, and sends it to the device. The device also collects initial data to generate the user's emotional profile.

[2280] Route learning and configuration

[2281] When a user starts shopping in a virtual store, the device sends its current location information to the server. The server uses a generative AI model to calculate the optimal route based on the current location information, destination information, the user's past behavior history, and specific requests. It generates haptic feedback data and sends it to the device. The device stores this data and provides haptic feedback when needed.

[2282] Emotion Recognition and Adaptive Feedback

[2283] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate. If the user is feeling stressed or anxious, the emotion engine adjusts the haptic feedback pattern and provides reassuring feedback.

[2284] Specific examples

[2285] A user wears smart glasses and visits a virtual store. While searching for a product in the store, the emotion engine detects the user's anxiety and sends the emotion data to the server. The server calculates the optimal haptic feedback pattern and sends it to the device. The device then provides vibration and audio feedback, reducing the user's stress.

[2286] Prompt Sentence Examples

[2287] It recognizes the user's emotions while selecting products in a virtual store and provides immediate relaxation feedback.

[2288] It detects the stress felt by users searching for a specific product in a virtual store and provides haptic feedback.

[2289] This system and method provide optimal haptic feedback based on the user's emotional state, enabling a comfortable shopping experience in a virtual store.

[2290] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2291] Step 1:

[2292] The user turns on the device and enters basic information (name, age, specific requests).

[2293] Specific operation: The user enters their name, age, and specific requests into the terminal, and then sends that information to the server by pressing the send button.

[2294] Input: User's basic information (name, age, specific needs).

[2295] Data processing / calculation: The basic information entered by the terminal is encrypted and sent to the server.

[2296] Output: Encrypted user data.

[2297] Step 2:

[2298] The server stores the received user information in a database, generates a user ID, and sends it to the terminal.

[2299] Specific operation: The server saves the received user information in a database, generates a new user ID, and returns the ID to the device.

[2300] Input: Encrypted user data.

[2301] Data processing / calculation: Saving user data and generating user IDs.

[2302] Output: The generated user ID.

[2303] Step 3:

[2304] The device collects initial data to generate an emotional profile of the user.

[2305] Specific operation: The device records initial emotion data using the camera, microphone, heart rate sensor, etc.

[2306] Input: User's voice, facial expressions, heart rate, etc.

[2307] Data processing / computation: Collecting and recording emotional data.

[2308] Output: Initial emotion profile data.

[2309] Step 4:

[2310] When a user starts shopping in a virtual store, the terminal transmits current location information to the server.

[2311] Specific operation: The device uses the GPS module to obtain its current location information and sends it to the server.

[2312] Input: The user's current location.

[2313] Data processing / calculation: Acquisition and transmission of location information.

[2314] Output: Current location information sent to the server.

[2315] Step 5:

[2316] The server uses a generative AI model to calculate the optimal route based on the current location, destination, the user's past behavior, and specific requests.

[2317] Specific operation: Based on the data received by the server, the generative AI model is used to calculate the optimal shopping route.

[2318] Input: current location, destination, past activity history, specific requests.

[2319] Data processing / calculation: Route calculation using generative AI models.

[2320] Output: Optimal route data.

[2321] Step 6:

[2322] Haptic feedback data is generated based on the optimal path and transmitted to the device.

[2323] Specific operation: The server generates haptic feedback data based on the route data calculated and sends it to the device.

[2324] Input: Optimal route data.

[2325] Data processing / computation: Generation of haptic feedback data.

[2326] Output: Haptic feedback data.

[2327] Step 7:

[2328] The terminal stores the received haptic feedback data and provides haptic feedback when necessary.

[2329] Specific operation: The device stores the received data in its internal storage and provides it to the user via the haptic feedback device when necessary.

[2330] Input: Haptic feedback data.

[2331] Data processing / calculation: Storing data and giving instructions to the haptic device.

[2332] Output: Haptic feedback.

[2333] Step 8:

[2334] The emotion engine recognizes emotions from sensor data such as the user's voice, facial expressions, and heart rate.

[2335] Specific operation: The device inputs data acquired by sensors into the emotion engine and analyzes the user's emotions in real time.

[2336] Input: speech, facial expression, and heart rate data.

[2337] Data processing / computation: Analysis of emotion data.

[2338] Output: User's emotional state data.

[2339] Step 9:

[2340] If the user is feeling stressed or anxious, the device sends that information to the server.

[2341] Specific operation: The terminal transmits the user's emotional state to the server, and the server receives it.

[2342] Input: Emotional state data.

[2343] Data processing / calculation: Sending and receiving emotional data.

[2344] Output: Emotion data sent to the server.

[2345] Step 10:

[2346] The server calculates the optimal haptic feedback pattern and sends it to the device.

[2347] Specific operation: Based on the emotional state, the server adjusts the haptic feedback, generates new patterns, and sends them to the device.

[2348] Input: Emotional state data.

[2349] Data processing / calculation: Generation of optimal haptic feedback patterns.

[2350] Output: New haptic feedback data.

[2351] Step 11:

[2352] The device provides new haptic feedback to the user, reducing stress and anxiety.

[2353] Specific operation: The device activates the haptic feedback device based on the new haptic feedback data, providing the user with a sense of security.

[2354] Input: New haptic feedback data.

[2355] Data processing / computation: Implementing haptic feedback.

[2356] Output: Improved user emotional state.

[2357] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2358] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2359] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2360] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2361] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2362] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2363] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2364] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2365] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2366] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2367] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2368] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2369] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2370] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2371] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2372] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2373] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2374] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2375] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2376] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2377] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2378] The following is further disclosed regarding the above embodiment.

[2379] (Claim 1)

[2380] A means for acquiring current location information of a user;

[2381] A means for acquiring destination information and route conditions designated by a user;

[2382] a means for calculating an optimal route based on the acquired current location information, destination information, and route conditions, and generating haptic feedback data;

[2383] means for providing the generated haptic feedback data to a user along with the haptic feedback;

[2384] A system including:

[2385] (Claim 2)

[2386] 10. The system of claim 1, further comprising means for calculating a corrective path and generating new haptic feedback data if the user is heading in the wrong direction.

[2387] (Claim 3)

[2388] 10. The system of claim 1, further comprising means for confirming that the destination has been reached and providing notification of the destination being reached to the user with haptic feedback.

[2389] (Claim 4)

[2390] 10. The system of claim 1, further comprising means for obtaining basic information about the user and generating and storing a unique identification number.

[2391] (Claim 5)

[2392] The system of claim 1, further comprising means for using a generative model to learn an optimal route based on the acquired basic information and in accordance with the user's past travel history and requests.

[2393] "Example 1"

[2394] (Claim 1)

[2395] A means for acquiring current location information of a user;

[2396] A means for acquiring destination information and travel conditions designated by a user;

[2397] A means for calculating an optimal route using a generative AI model based on the acquired current location information, destination information, and travel conditions, and generating haptic feedback data;

[2398] means for providing the generated haptic feedback data to a user along with the haptic feedback;

[2399] A system including:

[2400] (Claim 2)

[2401] 10. The system of claim 1, further comprising means for calculating a corrective path and generating new haptic feedback data if the user is heading in the wrong direction.

[2402] (Claim 3)

[2403] 10. The system of claim 1, further comprising means for confirming that the destination has been reached and providing notification of the destination being reached to the user with haptic feedback.

[2404] "Application Example 1"

[2405] (Claim 1)

[2406] A means for acquiring current location information of a user;

[2407] A means for acquiring user-specified target product information and in-store conditions;

[2408] a means for calculating an optimal route based on the acquired current location information, target product information, and in-store conditions, and generating haptic feedback data;

[2409] means for providing the generated haptic feedback data to a user along with the haptic feedback;

[2410] A system including:

[2411] (Claim 2)

[2412] 10. The system of claim 1, further comprising means for calculating a corrective path and generating new haptic feedback data if the user is heading in the wrong direction.

[2413] (Claim 3)

[2414] 10. The system of claim 1, further comprising means for confirming that the target item has been reached and providing notification of the arrival to the user with haptic feedback.

[2415] "Example 2: Combining Emotion Engines"

[2416] (Claim 1)

[2417] A means for acquiring current location information of a user;

[2418] means for acquiring destination information and requirements specified by a user;

[2419] A means for calculating an optimal route based on the acquired current location information, destination information, and requirements, and generating haptic feedback data using a generative AI model;

[2420] means for providing the generated haptic feedback data to a user along with the haptic feedback;

[2421] A means of collecting sensor data and recognizing user emotions in real time;

[2422] means for adjusting a haptic feedback pattern in response to a user's emotion;

[2423] A system including:

[2424] (Claim 2)

[2425] 10. The system of claim 1, further comprising means for calculating a corrective path and generating new haptic feedback data if the user is heading in the wrong direction.

[2426] (Claim 3)

[2427] 10. The system of claim 1, further comprising means for confirming that the destination has been reached and providing notification of the destination being reached to the user with haptic feedback.

[2428] "Application example 2 when combining emotion engines"

[2429] (Claim 1)

[2430] A means for acquiring current location information of a user;

[2431] A means for acquiring destination information and route conditions designated by a user;

[2432] a means for calculating an optimal route based on the acquired current location information, destination information, and route conditions, and generating haptic feedback data;

[2433] means for providing the generated haptic feedback data to a user along with the haptic feedback;

[2434] means for recognizing a user's emotions in real time and transmitting the data to a server;

[2435] means for receiving an optimal haptic feedback pattern from the server and providing it to the user;

[2436] A system including:

[2437] (Claim 2)

[2438] 10. The system of claim 1, further comprising means for calculating a corrective path and generating new haptic feedback data if the user is heading in the wrong direction.

[2439] (Claim 3)

[2440] 10. The system of claim 1, further comprising means for confirming that the destination has been reached and providing notification of the destination being...

Claims

1. A means for acquiring current location information of a user; A means for acquiring destination information and route conditions designated by a user; a means for calculating an optimal route based on the acquired current location information, destination information, and route conditions, and generating haptic feedback data; means for providing the generated haptic feedback data to a user along with the haptic feedback; A system including:

2. The system of claim 1 , further comprising means for calculating a corrective path and generating new haptic feedback data if the user is heading in the wrong direction.

3. The system of claim 1 further comprising means for confirming that the destination has been reached and providing notification of the destination reached to the user with haptic feedback.

4. 10. The system of claim 1, further comprising means for obtaining basic information about the user and generating and storing a unique identification number.

5. The system according to claim 1 , further comprising means for using a generative model to learn an optimal route according to the user's past travel history and requests based on the acquired basic information.

Images