system

Abstract

We provide the system. [Solution] Information acquisition means for obtaining location information and environmental information from the user, A data collection method for collecting information on the operation of public transportation, A route generation means that generates the optimal route based on acquired location information and traffic information, A guidance means that presents the generated route to the user via audio and visual means, A notification method that sends a notification to an emergency contact in the event of an abnormal situation, External information processing means intended to support individuals with mobility difficulties, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When an individual with a developmental disorder uses public transportation and faces different traffic situations than normal, it is necessary to improve the problem of falling into confusion or panic. Also, there is a need for a reliable navigation system to support their safe and smooth movement and promote social participation. Furthermore, an emergency notification system that allows people around them and guardians to respond quickly is required.

Means for Solving the Problems

[0005] This invention provides a location information acquisition means for acquiring the user's location information and surrounding environment information. This allows for accurate determination of the user's current location. Furthermore, it includes an information collection means for acquiring the latest traffic conditions by collecting real-time operation information of public transportation. It also includes a route generation means for generating an optimal travel route based on the acquired location information and traffic information. The generated travel route is presented to the user both audibly and visually. In addition, a notification means is set up to send notifications to emergency contacts in the event of an abnormal situation, enabling a rapid response. Furthermore, by acquiring environmental video data from public monitoring devices and utilizing it in travel route generation, safer and more accurate navigation is achieved. Finally, it includes a feedback processing means for collecting user feedback and continuously improving the system.

[0006] "Location information acquisition means" refers to devices or systems for accurately acquiring the user's current location and surrounding environmental information.

[0007] "Information gathering means" refers to devices and systems for efficiently collecting various types of information, including real-time operational data of public transportation.

[0008] A "route generation means" is a processing device or software that generates the optimal route for travel based on acquired location information and traffic information.

[0009] A "guidance means" refers to a device or method that presents generated route information to a user via voice or visual means.

[0010] A "notification mechanism" refers to a device or process for promptly and appropriately notifying pre-designated emergency contacts in the event of an emergency.

[0011] A "public surveillance device" is a device that uses cameras and sensors installed in public places to acquire video data of the surrounding environment.

[0012] A "feedback processing means" refers to a processing device or software that collects feedback provided by users and uses it to improve the operation of the system. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14]It is a sequence diagram showing the processing flow of a data processing system in Application Example 2 when a sentiment engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, a labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, a labeled storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is an integrated navigation system designed to support users with developmental disabilities in safely and efficiently using public transportation. The system configuration and its usage are described below.

[0035] First, the user carries the device and begins to move. The device uses GPS to determine the user's current location and transmits this information to the server through a location information acquisition method. The server calculates the route to the user's planned destination based on public transport operation information collected from a real-time traffic API. This information includes traffic congestion and delay information.

[0036] The server uses an AI algorithm to generate the optimal travel route and sends this route information to the terminal. The terminal visually displays the route information to the user and uses voice guidance to inform the user of the next action. This makes it easier for the user to intuitively understand the instructions.

[0037] In the event of a transportation disruption, accident, or other emergency, the server will immediately generate a new alternative route and notify the user via their device. Furthermore, in an emergency, notifications will be sent to pre-registered guardians or caregivers. This function supports emergency contact methods.

[0038] As a concrete example, consider a typical commute to school. If a user's planned train is cancelled during their commute, the server immediately retrieves this information and guides the device with walking directions to the nearest bus stop and the time of the next bus. This guidance is provided through a simple map and audio guide.

[0039] After the transfer is complete, users can provide feedback on their experience through their device. This feedback is collected on the server and used to continuously improve the system. This allows for flexible responses tailored to the individual needs and challenges of each user.

[0040] In this way, this system can provide support to enable users to go out and live social lives with peace of mind, and can also contribute to building an inclusive environment for society as a whole.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The user starts up the device and enables location services. This prepares the device to determine the user's current location.

[0044] Step 2:

[0045] The device uses GPS to obtain the user's precise location information and sends it to the server. This information serves as basic data necessary for subsequent processing.

[0046] Step 3:

[0047] The server accesses multiple transportation APIs to collect real-time information on public transport operations. It retrieves the operating status and delay information of each transportation service to understand the current traffic situation.

[0048] Step 4:

[0049] The server integrates location and traffic information and uses an AI algorithm to calculate the optimal travel route. Here, route selection is made considering both efficiency and safety.

[0050] Step 5:

[0051] The calculated travel route is sent to the terminal, which then presents the route information to the user. The display is done through a map format and voice guidance, designed to be simple and intuitive to understand.

[0052] Step 6:

[0053] While in transit, the server continuously monitors changes in traffic conditions. In the event of an emergency (such as a service cancellation or accident), a new alternative route is immediately generated.

[0054] Step 7:

[0055] If an abnormal situation occurs, the server sends a notification to the terminal, and the terminal informs the user. Additionally, notifications are sent to pre-registered emergency contacts as needed.

[0056] Step 8:

[0057] After the transfer is complete, the user provides feedback via their device. This feedback is collected by the server and used to improve future services.

[0058] (Example 1)

[0059] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0060] There is a need to provide an integrated system that alleviates the difficulties and anxieties faced by users with developmental disabilities when using public transportation, and supports safe and efficient travel. Furthermore, feedback mechanisms are necessary to continuously improve the travel experience for individual users.

[0061] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0062] In this invention, the server includes location information acquisition means for detecting and dynamically transmitting location information, information collection means for aggregating operational information in real time, and route generation means for designing optimized travel routes using AI algorithms. This enables users to travel safely and efficiently and respond quickly to abnormal situations.

[0063] A "location information acquisition means" is a technical device for accurately detecting the user's current location and transmitting it to a server.

[0064] "Information gathering means" refers to functions that efficiently acquire and aggregate real-time operational information of public transportation and make it available throughout the entire system.

[0065] "Route generation means" refers to the process of generating the optimal travel route for the user by utilizing AI algorithms based on obtained location information and traffic information.

[0066] A "guidance method" is a technology that provides users with generated travel routes visually and audibly, guiding them in an intuitively understandable way.

[0067] A "notification mechanism" is a function that provides an appropriate alternative route and quickly notifies emergency contacts of the situation when an abnormal situation occurs.

[0068] An "environmental monitoring device" is a device that collects video data of the surrounding environment and incorporates that information into the route generation process.

[0069] A "feedback processing mechanism" is a function that collects feedback data from users regarding their travel experience and uses it to improve the system.

[0070] Embodiments of this invention will now be described. This system is a navigation system designed to enable users with developmental disabilities to use public transportation safely and efficiently.

[0071] First, the device carried by the user is equipped with a GPS module, which allows the user's current location to be accurately determined. The device transmits this location information to a server. This transmission is carried out via mobile data communication or a Wi-Fi connection.

[0072] After receiving location information, the server uses data collection methods to obtain real-time public transport operation information from a public transport API. For example, it can use an API provided by a common transport data provider. The information obtained also includes data on congestion and delays.

[0073] The server utilizes AI algorithms to analyze acquired location and traffic information. By implementing deep learning or reinforcement learning algorithms using a generative AI model, it calculates the optimal travel route. The generated route includes detailed information such as walking routes, transfer information, and the duration of use for each mode of transport.

[0074] Next, the server sends route information to the terminal, which then guides the user through visual and audio guidance. The guidance is intuitive, using a dedicated map app and voice assistant, allowing the user to navigate accordingly.

[0075] As a concrete example, consider a scenario where a train service is suspended during the commute to school. The server retrieves the suspension information in real time and guides the user to an alternative route. For example, it might display a concise walking route to the nearest bus stop and the time of the next bus, along with voice guidance.

[0076] An example of a prompt message is: "Please tell me the best route from my current location to my destination. Public transportation is required."

[0077] This system allows users to respond flexibly to unexpected situations and travel with peace of mind.

[0078] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0079] Step 1:

[0080] The user carries the device and begins to move. The device uses its built-in GPS module to determine the user's current location. This input allows the device to obtain latitude and longitude coordinate data. This acquired location information is transmitted to the server via mobile data communication or Wi-Fi.

[0081] Step 2:

[0082] The server begins processing the location information received from the terminal. Next, the server uses a real-time traffic API to collect publicly available traffic information. Specifically, it obtains data that may affect the user's movement, such as congestion levels and delay information. This information is input to the server, creating an up-to-date traffic information database.

[0083] Step 3:

[0084] The server uses location and traffic information as input to perform data analysis using AI algorithms. Specifically, it optimizes routes using generative AI models. This is a process that generates the optimal travel route using deep learning models or reinforcement learning. As output, a travel route optimized for the user is generated.

[0085] Step 4:

[0086] The server sends the generated route information to the terminal. The terminal receives this as input and prepares directions for the user. The terminal uses a dedicated navigation app to visually display the route on a map. It also uses a voice assistant to notify the user of the next action by voice. As a result, the output provides the user with directions that are easy to understand intuitively.

[0087] Step 5:

[0088] In the event of a disruption in public transport, the server checks the latest traffic information and re-evaluates existing routes. Upon receiving the disruption information as input, the server immediately generates an alternative route. This new route information is sent to the terminal, and an emergency notification is issued to the user. As output, the user can obtain the new route guidance.

[0089] Step 6:

[0090] After the journey is complete, the user provides feedback via a terminal. The terminal collects the user's feedback data as input and sends it to the server. The server processes this feedback and uses it to continuously improve the entire system. As an output, the system's guidance functions and route generation algorithms are improved, resulting in a better user experience in the future.

[0091] (Application Example 1)

[0092] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0093] For individuals with developmental disabilities, the elderly, and others who have difficulty with mobility, there is a need to alleviate their anxieties about using public transportation to travel safely and efficiently to their destinations, and especially to enable smooth responses in the event of an emergency. In particular, since these individuals have difficulty independently resolving problems when faced with sudden changes in the environment or unpredictable troubles, the development of support systems is a challenge.

[0094] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0095] In this invention, the server includes information acquisition means for acquiring location information and environmental information, data collection means for collecting public transportation operation information, and route generation means for generating an optimal route based on the acquired location information and traffic information. This enables individuals with mobility difficulties to travel to their destinations safely when using public transportation, and also allows for a swift and appropriate response in the event of an emergency.

[0096] "Information acquisition means" refers to devices and technologies that accurately acquire the user's current location and surrounding environmental information.

[0097] "Data collection methods" refer to systems and techniques for collecting real-time information on public transportation and uploading it to a server.

[0098] "Route generation means" refers to a process or device for calculating and generating the optimal travel route based on collected location information and traffic information.

[0099] "Guidance means" refers to methods or devices that provide instructions to the user through audio or visual information based on a generated route.

[0100] "Notification means" refers to systems or technologies that automatically transmit information to pre-registered emergency contacts in the event of an emergency.

[0101] "External information processing means" refers to technologies and systems that process necessary external information for the purpose of supporting individuals with mobility difficulties.

[0102] "Environmental data" refers to information about external conditions that affect movement, such as the user's surroundings, weather, and traffic volume.

[0103] "Data processing means" refers to processes and devices used to analyze feedback information collected after movement and to improve the system.

[0104] The system for implementing this invention consists of a user-carried terminal, a server for processing data, and a network infrastructure for integrating these. First, the terminal uses a GPS module to obtain the user's current location. This location information is transmitted to the server in real time. Based on this information, the server collects public transport operation information via a real-time traffic API. Specifically, this includes weather, traffic congestion, and service delay information.

[0105] The server uses a route generation system to calculate the optimal travel route for the user based on the collected data. An AI algorithm is used for this calculation. The generated route information is sent to the terminal and presented to the user as audio and visual information by the terminal's guidance system. Text-to-speech software (e.g., Google® Text-to-Speech API) is used for the audio guidance.

[0106] If the terminal detects an abnormal situation, it immediately sends information to the server and regenerates an alternative optimal route. It also sends emergency notifications to registered contacts using notification methods as needed. During this process, the user's movement history and feedback information are stored on the server and used to improve the system.

[0107] As a concrete example, if a user encounters a train cancellation on their way to a museum, the server will immediately grasp this information and provide a walking route to the nearest bus stop and a timetable for the next bus.

[0108] An example of a prompt message would be, "The train the user was planning to take to the art museum has been cancelled. Please suggest an alternative route in this situation." This allows for flexible and real-time responses through the generative AI model.

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The terminal uses a GPS module to obtain the user's current location. It receives GPS data as input, generates location information as output, and sends it to the server. In this step, the terminal analyzes the GPS signal to obtain latitude and longitude data.

[0112] Step 2:

[0113] The server uses the received location information to collect public transport operation information using a real-time transportation API. As input, it makes requests to the API based on the location information, and as output, it obtains information such as operation status and delays. The server organizes this information and prepares it as basic data for creating optimal routes.

[0114] Step 3:

[0115] The server uses AI algorithms to generate optimal travel routes from location and traffic information. It takes well-organized location and traffic data as input, performs AI analysis, and creates optimal route data as output. This calculates efficient and safe routes for users.

[0116] Step 4:

[0117] The generated route information is sent from the server to the terminal. The terminal provides this route information to the user as audio and visual information. It receives route data from the server as input and displays navigation guides on the audio output device and display as output. Specifically, the terminal provides voice guidance using the Google Text-to-Speech API.

[0118] Step 5:

[0119] If an abnormal situation occurs, the terminal immediately sends the information to the server. The server regenerates a new, optimal route and sends it back to the terminal. In this step, the server again utilizes AI to recalculate a precise route based on the latest traffic conditions and, if necessary, sends notifications to emergency contacts via notification methods.

[0120] Step 6:

[0121] After the user finishes their journey, the terminal sends feedback information to the server. The server receives user experience evaluations as input and stores this data as output. The server uses this feedback data to improve the system. Specific actions include simple and intuitive feedback collection through the user interface.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention provides a navigation system incorporating an emotion engine to help users with developmental disabilities use public transportation with peace of mind. The system configuration and usage method are described below.

[0124] When a user picks up a device and begins to move, the device first determines its current location through a location information acquisition method. Then, the server obtains public transportation operation information from a transportation API to understand real-time traffic conditions. Based on this information, the optimal travel route is calculated.

[0125] The system is equipped with an emotion engine that senses the user's emotional state in real time. The device analyzes the user's facial expressions, tone of voice, and input data, and uses the emotion engine to analyze the user's emotional state. As a result, the navigation of the travel route is adjusted according to the user's stress level. For example, if anxiety is detected, the guidance is switched to a simpler and more reassuring expression.

[0126] One possible scenario is when a user feels anxious due to train delay information. Based on the latest service information received from the server, the device will guide the user to a preferred alternative route to encourage safer travel. Depending on the results of the emotion engine's analysis, voice guidance such as, "Please stay calm, I will guide you to the next bus stop," may also be provided.

[0127] Furthermore, this system uses a feedback processing mechanism to encourage friendly feedback after the move is completed, accumulating user experience points. User feedback is collected on the server and used to improve future services.

[0128] In this way, this system enhances users' sense of security when going out and traveling, thereby increasing opportunities for social participation and contributing to the improvement of users' quality of life (QOL).

[0129] The following describes the processing flow.

[0130] Step 1:

[0131] The user starts up their device and prepares to begin moving. At this stage, the emotion engine also starts up and begins measuring the user's initial emotional state.

[0132] Step 2:

[0133] The device uses GPS to obtain the user's current location. This location information is sent to the server, allowing for accurate understanding of the current environment.

[0134] Step 3:

[0135] The server retrieves real-time public transport service information via an API. This information includes data on congestion, delays, and cancellations.

[0136] Step 4:

[0137] The server uses an AI model to calculate the optimal travel route based on acquired location and operational information. It also considers the results of the emotion engine's analysis to select a stress-free route.

[0138] Step 5:

[0139] The system sends a calculated optimal travel route to the device, which then provides the user with corresponding visual navigation and voice guidance. The guidance is adjusted based on the user's emotional state.

[0140] Step 6:

[0141] While the user is on the move, the device's emotion engine continuously analyzes the user's voice and facial expressions, monitoring changes in their emotions. If an abnormal emotion is detected, the system immediately changes the guidance provided to reassure the user.

[0142] Step 7:

[0143] In the event of an emergency or traffic change, the server retrieves updated traffic information and recalculates alternative routes. This information is then sent to the terminal to guide the user to a safe travel route.

[0144] Step 8:

[0145] After the transfer is complete, the user can enter feedback and send information about their experience to their device. The feedback data is stored on the server and used for future guidance and service improvements.

[0146] (Example 2)

[0147] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0148] When users with developmental disabilities use public transportation, there is a need for a navigation system that provides optimal routes in real time according to traffic conditions, while also considering the user's emotional state and providing a sense of security. However, existing systems do not take the user's emotional state into account, and therefore fail to reduce anxiety and stress and provide a better travel experience.

[0149] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0150] In this invention, the server includes location information acquisition means for acquiring the user's own location information and surrounding situation information, information collection means for collecting real-time operation information of public transportation, route generation means for generating an optimal travel route based on the acquired location information and traffic information, and emotion analysis means for analyzing the user's emotional state and adjusting the guidance content based on that state. This makes it possible to provide customized travel guidance according to the user's real-time emotional state, giving the user a sense of security and realizing a more comfortable travel experience.

[0151] "Location information acquisition means" refers to a function for determining the user's current location, and typically involves a device that obtains latitude and longitude using technologies such as GPS.

[0152] "Information gathering means" refers to a function for acquiring data related to the operation of public transportation, and is a method for obtaining real-time operational information from transportation APIs.

[0153] A "route generation means" is a function that calculates the optimal travel route for the user based on collected location information and traffic information, and is a device that uses an algorithm to derive a route that takes time and distance into consideration.

[0154] A "guidance means" is a function that presents the generated travel route to the user visually or audibly, and is a device for appropriately conveying navigation instructions.

[0155] "Emotional analysis means" refers to a function that analyzes the user's emotional state from their facial expressions and tone of voice, and adjusts the navigation guidance based on the results obtained.

[0156] "Notification method" refers to a function that automatically sends notifications to pre-configured emergency contacts in the event of an abnormal situation.

[0157] This invention is a navigation system that enables users with developmental disabilities to use public transportation with peace of mind. The system includes means for acquiring location information, means for collecting information, means for generating routes, means for providing guidance, means for analyzing emotions, and means for providing notifications.

[0158] The device determines the user's current location using GPS-based location acquisition. The server utilizes a transportation API to collect real-time public transport operation information and calculates the optimal travel route based on that information. In route generation, an algorithm that prioritizes the shortest travel time is applied.

[0159] The device provides visual and auditory guidance along the generated travel path. An emotion analysis system recognizes the user's facial expressions and tone of voice to assess their emotional state and adjust the guidance accordingly. For example, if anxiety is detected, the guidance changes to a simpler, more reassuring style.

[0160] For example, if a user feels stressed due to congestion or delays at a train station, the system analyzes their emotional state and provides a preferred alternative route to help them relax. Based on the emotional analysis, a voice message will also play saying, "Please calm down, we will guide you to the best route to your next destination."

[0161] Furthermore, in the event of an emergency, a notification will be automatically sent to pre-configured emergency contacts via a notification system.

[0162] After the user completes their journey, the device collects feedback. The server uses this feedback to improve the service.

[0163] An example of a prompt that utilizes a generative AI model is: "Use the generative AI model to generate a natural language description of the emotion engine required for a public transportation user support system for users with developmental disabilities."

[0164] This system aims to provide navigation that is sensitive to the user's emotions, thereby creating a safer and more secure travel experience.

[0165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0166] Step 1:

[0167] The terminal uses a location information acquisition method to obtain its current location using GPS functionality. The input is the GPS satellite signal received by the terminal, and the output is location information including latitude and longitude. This location information is used as basic data for subsequent processing.

[0168] Step 2:

[0169] The server accesses a transportation API via an information gathering mechanism to obtain real-time public transport operation information. The input is an API key that provides the latest traffic conditions, and the output is operation information data in JSON format. This information provides an important element in route generation.

[0170] Step 3:

[0171] The server calculates the optimal travel route by combining location information and operational information. The input is the user's current location and real-time operational information, and the output is optimal route data that considers the shortest travel time and the fewest transfers. The algorithm used includes Dijkstra's algorithm. This calculation is essential for formulating a convenient travel plan for the user.

[0172] Step 4:

[0173] The device uses emotion analysis to capture the user's facial expressions and voice tone through the camera and microphone. The input is this raw data, and the output is an analysis result indicating the user's emotional state. This analysis result allows for adjustments to the guidance provided.

[0174] Step 5:

[0175] The device provides guidance to the user based on the generated travel route and the results of sentiment analysis. Inputs are route data and emotional state, while outputs include visual map displays and voice guidance. For example, if user anxiety is detected, the guidance method switches to a simpler, more reassuring approach.

[0176] Step 6:

[0177] The device receives feedback from the user after they complete their journey. Input consists of user comments in text or selection format, and output is data used to improve the service. This will enhance the convenience of the service for future visits.

[0178] Step 7:

[0179] The server analyzes feedback data to improve the entire system. The input is feedback data, and the output is the extraction of areas for improvement and new service proposals. This stage allows the system to continuously evolve and improve the user experience.

[0180] (Application Example 2)

[0181] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0182] When using public transportation, both users with developmental disabilities and general users often experience anxiety and stress regarding transportation information and travel routes. In particular, a lack of consideration for emotional states prevents users from traveling with peace of mind. Furthermore, there is a need for improved services, including emergency response and feedback after use.

[0183] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0184] In this invention, the server includes an information acquisition device for acquiring location information, an information collection device for collecting transportation operation information, and an emotion analysis device for analyzing the user's emotional state. This makes it possible to provide optimal travel route guidance and a sense of security based on the user's emotional state. Furthermore, continuous improvement of the service can be achieved by incorporating feedback after use.

[0185] An "information acquisition device" is a device that acquires location information of the user's current location and surrounding environment.

[0186] An "information gathering device" is a device that collects real-time operational information of public transportation.

[0187] A "route generation device" is a device that generates the optimal travel route based on acquired location information and traffic information.

[0188] A "guidance device" is a device that presents the generated travel route to the user using both audio and visual means.

[0189] A "communication device" is a device that sends notifications to emergency contacts in the event of an abnormal situation.

[0190] An "emotion analysis device" is a device that estimates a user's emotional state in real time by analyzing their facial expressions and voice.

[0191] A "guidance adjustment device" is a device that provides customized guidance according to the user's emotional state.

[0192] A "feedback processing device" is a device that collects feedback data from users and uses it to improve services.

[0193] The system for realizing this invention is a multi-functional navigation system designed to make using public transportation safer. The system primarily consists of a user terminal and a server for processing information. The specific configuration and processing flow of the system are described below.

[0194] The server has an information acquisition device that obtains location information from the user's smartphone or other device. This device uses technologies such as GPS to determine the user's current location. The server then uses an information collection device that collects real-time operation information of public transportation to obtain the latest traffic information through a traffic API.

[0195] Next, the server uses this real-time information to generate the optimal travel route for the user using a route generation device. The software used includes a route optimization tool that employs AI algorithms. The generated route is presented to the user's terminal via a guidance device, both audibly and visually.

[0196] Furthermore, the terminal is equipped with an emotion analysis device that analyzes the user's facial expressions and tone of voice to estimate their emotional state in real time. This uses machine learning frameworks such as TENSORFLOW® Lite. Based on the analyzed emotional state, the guidance adjustment device adjusts the format and content of the guidance to provide appropriate feedback to the user.

[0197] For example, if a user feels anxious about the crowded conditions at a train station, an emotion analysis device will detect this state, and the guidance adjustment device will offer reassuring guidance such as, "Why don't you stop by a nearby cafe to calm down for a bit?"

[0198] In addition, after the user's journey is complete, a feedback processing unit collects feedback from the user and uses it to improve the service. This feedback is aggregated on the server side and used to improve the service in the future.

[0199] An example of a prompt message is: "Design a traffic navigation app using an emotion engine and provide real-time guidance that responds to the user's emotions."

[0200] In this way, the system will be attentive to the user's emotions and help create a society where people can travel with a sense of security.

[0201] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0202] Step 1:

[0203] The server receives a request from the user's terminal and obtains location information via an information acquisition device. The input is the user's location data, and the output is specific latitude and longitude information. Based on this information, the server records the user's location in its database.

[0204] Step 2:

[0205] The server uses a traffic API to obtain real-time operational information from data collection devices. The input is data provided by the traffic API, and the output is real-time information showing the operational status of public transportation. This information is stored in a database and organized into a format usable for subsequent route generation.

[0206] Step 3:

[0207] The server uses acquired location and traffic information to calculate the optimal travel route using a route generation device. The input is the user's location information and real-time traffic data, and the output is the recommended travel route for the user. An AI algorithm is used in the calculation, considering the shortest travel time and routes that avoid congestion.

[0208] Step 4:

[0209] The terminal receives route data transmitted from the server and presents it to the user audibly and visually using a guidance device. The input is the route data received from the server, and the output is audio guidance and map display. The information is displayed in a way that is easy for the user to understand.

[0210] Step 5:

[0211] The device analyzes the user's facial expressions and voice in real time using an emotion analysis device. Input is sensory information from the smartphone's camera and microphone, and output is the emotional state based on the analysis. A TensorFlow Lite model is used to determine the user's emotional state.

[0212] Step 6:

[0213] The server receives the results of the emotion analysis and uses a guidance adjustment device to adjust the guidance content according to the user's emotional state. The input is the user's emotional state data, and the output is a customized guidance message. If the user is feeling anxious, the content will be changed to provide reassurance.

[0214] Step 7:

[0215] After the user reaches their destination, a feedback processing device is used to collect feedback about the service at the terminal and send it to the server. The input is feedback data from the user, and the output is feedback information stored on the server. The feedback is used to improve the service in the future.

[0216] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0217] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0218] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0219] [Second Embodiment]

[0220] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0221] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0222] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0223] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0224] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0225] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0226] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0227] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0228] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0229] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0230] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0231] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0232] This invention is an integrated navigation system designed to support users with developmental disabilities in safely and efficiently using public transportation. The system configuration and its usage are described below.

[0233] First, the user carries the device and begins to move. The device uses GPS to determine the user's current location and transmits this information to the server through a location information acquisition method. The server calculates the route to the user's planned destination based on public transport operation information collected from a real-time traffic API. This information includes traffic congestion and delay information.

[0234] The server uses an AI algorithm to generate the optimal travel route and sends this route information to the terminal. The terminal visually displays the route information to the user and uses voice guidance to inform the user of the next action. This makes it easier for the user to intuitively understand the instructions.

[0235] In the event of a transportation disruption, accident, or other emergency, the server will immediately generate a new alternative route and notify the user via their device. Furthermore, in an emergency, notifications will be sent to pre-registered guardians or caregivers. This function supports emergency contact methods.

[0236] As a concrete example, consider a typical commute to school. If a user's planned train is cancelled during their commute, the server immediately retrieves this information and guides the device with walking directions to the nearest bus stop and the time of the next bus. This guidance is provided through a simple map and audio guide.

[0237] After the transfer is complete, users can provide feedback on their experience through their device. This feedback is collected on the server and used to continuously improve the system. This allows for flexible responses tailored to the individual needs and challenges of each user.

[0238] In this way, this system can provide support to enable users to go out and live social lives with peace of mind, and can also contribute to building an inclusive environment for society as a whole.

[0239] The following describes the processing flow.

[0240] Step 1:

[0241] The user starts up the device and enables location services. This prepares the device to determine the user's current location.

[0242] Step 2:

[0243] The device uses GPS to obtain the user's precise location information and sends it to the server. This information serves as basic data necessary for subsequent processing.

[0244] Step 3:

[0245] The server accesses multiple transportation APIs to collect real-time information on public transport operations. It retrieves the operating status and delay information of each transportation service to understand the current traffic situation.

[0246] Step 4:

[0247] The server integrates location and traffic information and uses an AI algorithm to calculate the optimal travel route. Here, route selection is made considering both efficiency and safety.

[0248] Step 5:

[0249] The calculated travel route is sent to the terminal, which then presents the route information to the user. The display is done through a map format and voice guidance, designed to be simple and intuitive to understand.

[0250] Step 6:

[0251] While in transit, the server continuously monitors changes in traffic conditions. In the event of an emergency (such as a service cancellation or accident), a new alternative route is immediately generated.

[0252] Step 7:

[0253] If an abnormal situation occurs, the server sends a notification to the terminal, and the terminal informs the user. Additionally, notifications are sent to pre-registered emergency contacts as needed.

[0254] Step 8:

[0255] After the transfer is complete, the user provides feedback via their device. This feedback is collected by the server and used to improve future services.

[0256] (Example 1)

[0257] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0258] There is a need to provide an integrated system that alleviates the difficulties and anxieties faced by users with developmental disabilities when using public transportation, and supports safe and efficient travel. Furthermore, feedback mechanisms are necessary to continuously improve the travel experience for individual users.

[0259] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0260] In this invention, the server includes location information acquisition means for detecting and dynamically transmitting location information, information collection means for aggregating operational information in real time, and route generation means for designing optimized travel routes using AI algorithms. This enables users to travel safely and efficiently and respond quickly to abnormal situations.

[0261] A "location information acquisition means" is a technical device for accurately detecting the user's current location and transmitting it to a server.

[0262] "Information gathering means" refers to functions that efficiently acquire and aggregate real-time operational information of public transportation and make it available throughout the entire system.

[0263] "Route generation means" refers to the process of generating the optimal travel route for the user by utilizing AI algorithms based on obtained location information and traffic information.

[0264] A "guidance method" is a technology that provides users with generated travel routes visually and audibly, guiding them in an intuitively understandable way.

[0265] A "notification mechanism" is a function that provides an appropriate alternative route and quickly notifies emergency contacts of the situation when an abnormal situation occurs.

[0266] An "environmental monitoring device" is a device that collects video data of the surrounding environment and incorporates that information into the route generation process.

[0267] A "feedback processing mechanism" is a function that collects feedback data from users regarding their travel experience and uses it to improve the system.

[0268] Embodiments of this invention will now be described. This system is a navigation system designed to enable users with developmental disabilities to use public transportation safely and efficiently.

[0269] First, the device carried by the user is equipped with a GPS module, which allows the user's current location to be accurately determined. The device transmits this location information to a server. This transmission is carried out via mobile data communication or a Wi-Fi connection.

[0270] After receiving location information, the server uses data collection methods to obtain real-time public transport operation information from a public transport API. For example, it can use an API provided by a common transport data provider. The information obtained also includes data on congestion and delays.

[0271] The server utilizes AI algorithms to analyze acquired location and traffic information. By implementing deep learning or reinforcement learning algorithms using a generative AI model, it calculates the optimal travel route. The generated route includes detailed information such as walking routes, transfer information, and the duration of use for each mode of transport.

[0272] Next, the server sends route information to the terminal, which then guides the user through visual and audio guidance. The guidance is intuitive, using a dedicated map app and voice assistant, allowing the user to navigate accordingly.

[0273] As a concrete example, consider a scenario where a train service is suspended during the commute to school. The server retrieves the suspension information in real time and guides the user to an alternative route. For example, it might display a concise walking route to the nearest bus stop and the time of the next bus, along with voice guidance.

[0274] An example of a prompt message is: "Please tell me the best route from my current location to my destination. Public transportation is required."

[0275] This system allows users to respond flexibly to unexpected situations and travel with peace of mind.

[0276] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0277] Step 1:

[0278] The user carries the device and begins to move. The device uses its built-in GPS module to determine the user's current location. This input allows the device to obtain latitude and longitude coordinate data. This acquired location information is transmitted to the server via mobile data communication or Wi-Fi.

[0279] Step 2:

[0280] The server begins processing the location information received from the terminal. Next, the server uses a real-time traffic API to collect publicly available traffic information. Specifically, it obtains data that may affect the user's movement, such as congestion levels and delay information. This information is input to the server, creating an up-to-date traffic information database.

[0281] Step 3:

[0282] The server performs data analysis using AI algorithms with location information and traffic information as inputs. Specifically, it optimizes the route by leveraging a generated AI model. This is a process of generating an optimal travel route using a deep learning model or reinforcement learning. As output, an optimized travel route for the user is generated.

[0283] Step 4:

[0284] The server sends the generated travel route information to the terminal. The terminal receives this as input and prepares guidance for the user. The terminal uses a dedicated navigation app to visually display the route on a map. Also, it uses a voice assistant to notify the user of the next action audibly. As a result, guidance that is intuitively easy for the user to understand is provided as output.

[0285] Step 5:

[0286] As a process for when an abnormality occurs in the transportation agency, the server checks the latest traffic information and re-evaluates the existing route. The server that receives the abnormality information as input immediately generates an alternative route. This new route information is sent to the terminal, and an emergency notification is sent to the user. As output, the user can obtain a new route guidance.

[0287] Step 6:

[0288] After the movement is completed, the user provides feedback via the terminal. The terminal collects the feedback data from the user as input and sends it to the server. The server processes this feedback and utilizes the feedback for the continuous improvement of the entire system. As output, the guidance function of the system and the route generation algorithm are improved, and the usage experience for subsequent times is enhanced.

[0289] (Application Example 1)

[0290] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0291] For individuals with developmental disabilities, the elderly, and others who have difficulty with mobility, there is a need to alleviate their anxieties about using public transportation to travel safely and efficiently to their destinations, and especially to enable smooth responses in the event of an emergency. In particular, since these individuals have difficulty independently resolving problems when faced with sudden changes in the environment or unpredictable troubles, the development of support systems is a challenge.

[0292] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0293] In this invention, the server includes information acquisition means for acquiring location information and environmental information, data collection means for collecting public transportation operation information, and route generation means for generating an optimal route based on the acquired location information and traffic information. This enables individuals with mobility difficulties to travel to their destinations safely when using public transportation, and also allows for a swift and appropriate response in the event of an emergency.

[0294] "Information acquisition means" refers to devices and technologies that accurately acquire the user's current location and surrounding environmental information.

[0295] "Data collection methods" refer to systems and techniques for collecting real-time information on public transportation and uploading it to a server.

[0296] "Route generation means" refers to a process or device for calculating and generating the optimal travel route based on collected location information and traffic information.

[0297] "Guidance means" refers to methods or devices that provide instructions to the user through audio or visual information based on a generated route.

[0298] "Notification means" refers to systems or technologies that automatically transmit information to pre-registered emergency contacts in the event of an emergency.

[0299] "External information processing means" refers to technologies and systems that process necessary external information for the purpose of supporting individuals with mobility difficulties.

[0300] "Environmental data" refers to information about external conditions that affect movement, such as the user's surroundings, weather, and traffic volume.

[0301] "Data processing means" refers to processes and devices used to analyze feedback information collected after movement and to improve the system.

[0302] The system for implementing this invention consists of a user-carried terminal, a server for processing data, and a network infrastructure for integrating these. First, the terminal uses a GPS module to obtain the user's current location. This location information is transmitted to the server in real time. Based on this information, the server collects public transport operation information via a real-time traffic API. Specifically, this includes weather, traffic congestion, and service delay information.

[0303] The server uses a route generation system to calculate the optimal travel route for the user based on the collected data. An AI algorithm is used for this calculation. The generated route information is sent to the terminal and presented to the user as audio and visual information by the terminal's guidance system. Text-to-speech software (e.g., Google Text-to-Speech API) is used for the audio guidance.

[0304] If the terminal detects an abnormal situation, it immediately sends information to the server and regenerates an alternative optimal route. It also sends emergency notifications to registered contacts using notification methods as needed. During this process, the user's movement history and feedback information are stored on the server and used to improve the system.

[0305] As a specific example, when a user encounters a train service disruption on the way to an art museum, the server immediately grasps this information and provides a walking route to the nearest bus stop and the timetable of the next bus to arrive.

[0306] As an example of a prompt sentence, in the form of "The train that the user had planned to take on the way to the art museum has been cancelled. Please consider a new route in this situation.", flexible and real-time responses can be achieved through the generative AI model.

[0307] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0308] Step 1:

[0309] The terminal uses the GPS module to obtain the user's current location. As input, it receives GPS data, generates the location information as output, and transmits it to the server. In this step, the terminal performs the operation of analyzing the GPS signal to obtain latitude and longitude data.

[0310] Step 2:

[0311] Based on the received location information, the server uses the real-time traffic API to collect the operation information of public transportation. As input, it makes a request to the API based on the location information, and as output, it obtains operation status, delay information, etc. The server organizes this information and prepares it as basic data for creating an optimal route.

[0312] Step 3:

[0313] The server uses an AI algorithm to generate an optimal moving route from the location information and traffic information. As input, it takes in the organized location information and traffic data, performs AI analysis, and as output, creates data for the optimal route. This calculates an efficient and safe route for the user.

[0314] Step 4:

[0315] The generated route information is sent from the server to the terminal. The terminal provides this route information to the user as audio and visual information. It receives route data from the server as input and displays navigation guides on the audio output device and display as output. Specifically, the terminal provides voice guidance using the Google Text-to-Speech API.

[0316] Step 5:

[0317] If an abnormal situation occurs, the terminal immediately sends the information to the server. The server regenerates a new, optimal route and sends it back to the terminal. In this step, the server again utilizes AI to recalculate a precise route based on the latest traffic conditions and, if necessary, sends notifications to emergency contacts via notification methods.

[0318] Step 6:

[0319] After the user finishes their journey, the terminal sends feedback information to the server. The server receives user experience evaluations as input and stores this data as output. The server uses this feedback data to improve the system. Specific actions include simple and intuitive feedback collection through the user interface.

[0320] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0321] This invention provides a navigation system incorporating an emotion engine to help users with developmental disabilities use public transportation with peace of mind. The system configuration and usage method are described below.

[0322] When a user picks up a device and begins to move, the device first determines its current location through a location information acquisition method. Then, the server obtains public transportation operation information from a transportation API to understand real-time traffic conditions. Based on this information, the optimal travel route is calculated.

[0323] The system is equipped with an emotion engine that senses the user's emotional state in real time. The device analyzes the user's facial expressions, tone of voice, and input data, and uses the emotion engine to analyze the user's emotional state. As a result, the navigation of the travel route is adjusted according to the user's stress level. For example, if anxiety is detected, the guidance is switched to a simpler and more reassuring expression.

[0324] One possible scenario is when a user feels anxious due to train delay information. Based on the latest service information received from the server, the device will guide the user to a preferred alternative route to encourage safer travel. Depending on the results of the emotion engine's analysis, voice guidance such as, "Please stay calm, I will guide you to the next bus stop," may also be provided.

[0325] Furthermore, this system uses a feedback processing mechanism to encourage friendly feedback after the move is completed, accumulating user experience points. User feedback is collected on the server and used to improve future services.

[0326] In this way, this system enhances users' sense of security when going out and traveling, thereby increasing opportunities for social participation and contributing to the improvement of users' quality of life (QOL).

[0327] The following describes the processing flow.

[0328] Step 1:

[0329] The user starts up their device and prepares to begin moving. At this stage, the emotion engine also starts up and begins measuring the user's initial emotional state.

[0330] Step 2:

[0331] The device uses GPS to obtain the user's current location. This location information is sent to the server, allowing for accurate understanding of the current environment.

[0332] Step 3:

[0333] The server retrieves real-time public transport service information via an API. This information includes data on congestion, delays, and cancellations.

[0334] Step 4:

[0335] The server uses an AI model to calculate the optimal travel route based on acquired location and operational information. It also considers the results of the emotion engine's analysis to select a stress-free route.

[0336] Step 5:

[0337] The system sends a calculated optimal travel route to the device, which then provides the user with corresponding visual navigation and voice guidance. The guidance is adjusted based on the user's emotional state.

[0338] Step 6:

[0339] While the user is on the move, the device's emotion engine continuously analyzes the user's voice and facial expressions, monitoring changes in their emotions. If an abnormal emotion is detected, the system immediately changes the guidance provided to reassure the user.

[0340] Step 7:

[0341] In the event of an emergency or traffic change, the server retrieves updated traffic information and recalculates alternative routes. This information is then sent to the terminal to guide the user to a safe travel route.

[0342] Step 8:

[0343] After the transfer is complete, the user can enter feedback and send information about their experience to their device. The feedback data is stored on the server and used for future guidance and service improvements.

[0344] (Example 2)

[0345] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0346] When users with developmental disabilities use public transportation, there is a need for a navigation system that provides optimal routes in real time according to traffic conditions, while also considering the user's emotional state and providing a sense of security. However, existing systems do not take the user's emotional state into account, and therefore fail to reduce anxiety and stress and provide a better travel experience.

[0347] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0348] In this invention, the server includes location information acquisition means for acquiring the user's own location information and surrounding situation information, information collection means for collecting real-time operation information of public transportation, route generation means for generating an optimal travel route based on the acquired location information and traffic information, and emotion analysis means for analyzing the user's emotional state and adjusting the guidance content based on that state. This makes it possible to provide customized travel guidance according to the user's real-time emotional state, giving the user a sense of security and realizing a more comfortable travel experience.

[0349] "Location information acquisition means" refers to a function for determining the user's current location, and typically involves a device that obtains latitude and longitude using technologies such as GPS.

[0350] "Information gathering means" refers to a function for acquiring data related to the operation of public transportation, and is a method for obtaining real-time operational information from transportation APIs.

[0351] A "route generation means" is a function that calculates the optimal travel route for the user based on collected location information and traffic information, and is a device that uses an algorithm to derive a route that takes time and distance into consideration.

[0352] A "guidance means" is a function that presents the generated travel route to the user visually or audibly, and is a device for appropriately conveying navigation instructions.

[0353] "Emotional analysis means" refers to a function that analyzes the user's emotional state from their facial expressions and tone of voice, and adjusts the navigation guidance based on the results obtained.

[0354] "Notification method" refers to a function that automatically sends notifications to pre-configured emergency contacts in the event of an abnormal situation.

[0355] This invention is a navigation system that enables users with developmental disabilities to use public transportation with peace of mind. The system includes means for acquiring location information, means for collecting information, means for generating routes, means for providing guidance, means for analyzing emotions, and means for providing notifications.

[0356] The device determines the user's current location using GPS-based location acquisition. The server utilizes a transportation API to collect real-time public transport operation information and calculates the optimal travel route based on that information. In route generation, an algorithm that prioritizes the shortest travel time is applied.

[0357] The device provides visual and auditory guidance along the generated travel path. An emotion analysis system recognizes the user's facial expressions and tone of voice to assess their emotional state and adjust the guidance accordingly. For example, if anxiety is detected, the guidance changes to a simpler, more reassuring style.

[0358] For example, if a user feels stressed due to congestion or delays at a train station, the system analyzes their emotional state and provides a preferred alternative route to help them relax. Based on the emotional analysis, a voice message will also play saying, "Please calm down, we will guide you to the best route to your next destination."

[0359] Furthermore, in the event of an emergency, a notification will be automatically sent to pre-configured emergency contacts via a notification system.

[0360] After the user completes their journey, the device collects feedback. The server uses this feedback to improve the service.

[0361] An example of a prompt that utilizes a generative AI model is: "Use the generative AI model to generate a natural language description of the emotion engine required for a public transportation user support system for users with developmental disabilities."

[0362] This system aims to provide navigation that is sensitive to the user's emotions, thereby creating a safer and more secure travel experience.

[0363] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0364] Step 1:

[0365] The terminal uses a location information acquisition method to obtain its current location using GPS functionality. The input is the GPS satellite signal received by the terminal, and the output is location information including latitude and longitude. This location information is used as basic data for subsequent processing.

[0366] Step 2:

[0367] The server accesses a transportation API via an information gathering mechanism to obtain real-time public transport operation information. The input is an API key that provides the latest traffic conditions, and the output is operation information data in JSON format. This information provides an important element in route generation.

[0368] Step 3:

[0369] The server calculates the optimal travel route by combining location information and operational information. The input is the user's current location and real-time operational information, and the output is optimal route data that considers the shortest travel time and the fewest transfers. The algorithm used includes Dijkstra's algorithm. This calculation is essential for formulating a convenient travel plan for the user.

[0370] Step 4:

[0371] The device uses emotion analysis to capture the user's facial expressions and voice tone through the camera and microphone. The input is this raw data, and the output is an analysis result indicating the user's emotional state. This analysis result allows for adjustments to the guidance provided.

[0372] Step 5:

[0373] The device provides guidance to the user based on the generated travel route and the results of sentiment analysis. Inputs are route data and emotional state, while outputs include visual map displays and voice guidance. For example, if user anxiety is detected, the guidance method switches to a simpler, more reassuring approach.

[0374] Step 6:

[0375] The device receives feedback from the user after they complete their journey. Input consists of user comments in text or selection format, and output is data used to improve the service. This will enhance the convenience of the service for future visits.

[0376] Step 7:

[0377] The server analyzes feedback data to improve the entire system. The input is feedback data, and the output is the extraction of areas for improvement and new service proposals. This stage allows the system to continuously evolve and improve the user experience.

[0378] (Application Example 2)

[0379] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0380] When using public transportation, both users with developmental disabilities and general users often experience anxiety and stress regarding transportation information and travel routes. In particular, a lack of consideration for emotional states prevents users from traveling with peace of mind. Furthermore, there is a need for improved services, including emergency response and feedback after use.

[0381] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0382] In this invention, the server includes an information acquisition device for acquiring location information, an information collection device for collecting transportation operation information, and an emotion analysis device for analyzing the user's emotional state. This makes it possible to provide optimal travel route guidance and a sense of security based on the user's emotional state. Furthermore, continuous improvement of the service can be achieved by incorporating feedback after use.

[0383] An "information acquisition device" is a device that acquires location information of the user's current location and surrounding environment.

[0384] An "information gathering device" is a device that collects real-time operational information of public transportation.

[0385] A "route generation device" is a device that generates the optimal travel route based on acquired location information and traffic information.

[0386] A "guidance device" is a device that presents the generated travel route to the user using both audio and visual means.

[0387] A "communication device" is a device that sends notifications to emergency contacts in the event of an abnormal situation.

[0388] An "emotion analysis device" is a device that estimates a user's emotional state in real time by analyzing their facial expressions and voice.

[0389] A "guidance adjustment device" is a device that provides customized guidance according to the user's emotional state.

[0390] A "feedback processing device" is a device that collects feedback data from users and uses it to improve services.

[0391] The system for realizing this invention is a multi-functional navigation system designed to make using public transportation safer. The system primarily consists of a user terminal and a server for processing information. The specific configuration and processing flow of the system are described below.

[0392] The server has an information acquisition device that obtains location information from the user's smartphone or other device. This device uses technologies such as GPS to determine the user's current location. The server then uses an information collection device that collects real-time operation information of public transportation to obtain the latest traffic information through a traffic API.

[0393] Next, the server uses this real-time information to generate the optimal travel route for the user using a route generation device. The software used includes a route optimization tool that employs AI algorithms. The generated route is presented to the user's terminal via a guidance device, both audibly and visually.

[0394] Furthermore, the device is equipped with an emotion analysis system that analyzes the user's facial expressions and tone of voice to estimate their emotional state in real time. This uses machine learning frameworks such as TensorFlow Lite. Based on the analyzed emotional state, the guidance adjustment system adjusts the format and content of the guidance to provide appropriate feedback to the user.

[0395] For example, if a user feels anxious about the crowded conditions at a train station, an emotion analysis device will detect this state, and the guidance adjustment device will offer reassuring guidance such as, "Why don't you stop by a nearby cafe to calm down for a bit?"

[0396] In addition, after the user's journey is complete, a feedback processing unit collects feedback from the user and uses it to improve the service. This feedback is aggregated on the server side and used to improve the service in the future.

[0397] An example of a prompt message is: "Design a traffic navigation app using an emotion engine and provide real-time guidance that responds to the user's emotions."

[0398] In this way, the system will be attentive to the user's emotions and help create a society where people can travel with a sense of security.

[0399] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0400] Step 1:

[0401] The server receives a request from the user's terminal and obtains location information via an information acquisition device. The input is the user's location data, and the output is specific latitude and longitude information. Based on this information, the server records the user's location in its database.

[0402] Step 2:

[0403] The server uses a traffic API to obtain real-time operational information from data collection devices. The input is data provided by the traffic API, and the output is real-time information showing the operational status of public transportation. This information is stored in a database and organized into a format usable for subsequent route generation.

[0404] Step 3:

[0405] The server uses acquired location and traffic information to calculate the optimal travel route using a route generation device. The input is the user's location information and real-time traffic data, and the output is the recommended travel route for the user. An AI algorithm is used in the calculation, considering the shortest travel time and routes that avoid congestion.

[0406] Step 4:

[0407] The terminal receives route data transmitted from the server and presents it to the user audibly and visually using a guidance device. The input is the route data received from the server, and the output is audio guidance and map display. The information is displayed in a way that is easy for the user to understand.

[0408] Step 5:

[0409] The device analyzes the user's facial expressions and voice in real time using an emotion analysis device. Input is sensory information from the smartphone's camera and microphone, and output is the emotional state based on the analysis. A TensorFlow Lite model is used to determine the user's emotional state.

[0410] Step 6:

[0411] The server receives the results of the emotion analysis and uses a guidance adjustment device to adjust the guidance content according to the user's emotional state. The input is the user's emotional state data, and the output is a customized guidance message. If the user is feeling anxious, the content will be changed to provide reassurance.

[0412] Step 7:

[0413] After the user reaches their destination, a feedback processing device is used to collect feedback about the service at the terminal and send it to the server. The input is feedback data from the user, and the output is feedback information stored on the server. The feedback is used to improve the service in the future.

[0414] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0415] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0416] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0417] [Third Embodiment]

[0418] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0419] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0420] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0421] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0422] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0423] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0424] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0425] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0426] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0427] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0428] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0429] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0430] This invention is an integrated navigation system designed to support users with developmental disabilities in safely and efficiently using public transportation. The system configuration and its usage are described below.

[0431] First, the user carries the device and begins to move. The device uses GPS to determine the user's current location and transmits this information to the server through a location information acquisition method. The server calculates the route to the user's planned destination based on public transport operation information collected from a real-time traffic API. This information includes traffic congestion and delay information.

[0432] The server uses an AI algorithm to generate the optimal travel route and sends this route information to the terminal. The terminal visually displays the route information to the user and uses voice guidance to inform the user of the next action. This makes it easier for the user to intuitively understand the instructions.

[0433] In the event of a transportation disruption, accident, or other emergency, the server will immediately generate a new alternative route and notify the user via their device. Furthermore, in an emergency, notifications will be sent to pre-registered guardians or caregivers. This function supports emergency contact methods.

[0434] As a concrete example, consider a typical commute to school. If a user's planned train is cancelled during their commute, the server immediately retrieves this information and guides the device with walking directions to the nearest bus stop and the time of the next bus. This guidance is provided through a simple map and audio guide.

[0435] After the transfer is complete, users can provide feedback on their experience through their device. This feedback is collected on the server and used to continuously improve the system. This allows for flexible responses tailored to the individual needs and challenges of each user.

[0436] In this way, this system can provide support to enable users to go out and live social lives with peace of mind, and can also contribute to building an inclusive environment for society as a whole.

[0437] The following describes the processing flow.

[0438] Step 1:

[0439] The user starts up the device and enables location services. This prepares the device to determine the user's current location.

[0440] Step 2:

[0441] The device uses GPS to obtain the user's precise location information and sends it to the server. This information serves as basic data necessary for subsequent processing.

[0442] Step 3:

[0443] The server accesses multiple transportation APIs to collect real-time information on public transport operations. It retrieves the operating status and delay information of each transportation service to understand the current traffic situation.

[0444] Step 4:

[0445] The server integrates location and traffic information and uses an AI algorithm to calculate the optimal travel route. Here, route selection is made considering both efficiency and safety.

[0446] Step 5:

[0447] The calculated travel route is sent to the terminal, which then presents the route information to the user. The display is done through a map format and voice guidance, designed to be simple and intuitive to understand.

[0448] Step 6:

[0449] While in transit, the server continuously monitors changes in traffic conditions. In the event of an emergency (such as a service cancellation or accident), a new alternative route is immediately generated.

[0450] Step 7:

[0451] If an abnormal situation occurs, the server sends a notification to the terminal, and the terminal informs the user. Additionally, notifications are sent to pre-registered emergency contacts as needed.

[0452] Step 8:

[0453] After the transfer is complete, the user provides feedback via their device. This feedback is collected by the server and used to improve future services.

[0454] (Example 1)

[0455] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0456] There is a need to provide an integrated system that alleviates the difficulties and anxieties faced by users with developmental disabilities when using public transportation, and supports safe and efficient travel. Furthermore, feedback mechanisms are necessary to continuously improve the travel experience for individual users.

[0457] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0458] In this invention, the server includes location information acquisition means for detecting and dynamically transmitting location information, information collection means for aggregating operational information in real time, and route generation means for designing optimized travel routes using AI algorithms. This enables users to travel safely and efficiently and respond quickly to abnormal situations.

[0459] A "location information acquisition means" is a technical device for accurately detecting the user's current location and transmitting it to a server.

[0460] "Information gathering means" refers to functions that efficiently acquire and aggregate real-time operational information of public transportation and make it available throughout the entire system.

[0461] "Route generation means" refers to the process of generating the optimal travel route for the user by utilizing AI algorithms based on obtained location information and traffic information.

[0462] A "guidance method" is a technology that provides users with generated travel routes visually and audibly, guiding them in an intuitively understandable way.

[0463] A "notification mechanism" is a function that provides an appropriate alternative route and quickly notifies emergency contacts of the situation when an abnormal situation occurs.

[0464] An "environmental monitoring device" is a device that collects video data of the surrounding environment and incorporates that information into the route generation process.

[0465] A "feedback processing mechanism" is a function that collects feedback data from users regarding their travel experience and uses it to improve the system.

[0466] Embodiments of this invention will now be described. This system is a navigation system designed to enable users with developmental disabilities to use public transportation safely and efficiently.

[0467] First, the device carried by the user is equipped with a GPS module, which allows the user's current location to be accurately determined. The device transmits this location information to a server. This transmission is carried out via mobile data communication or a Wi-Fi connection.

[0468] After receiving location information, the server uses data collection methods to obtain real-time public transport operation information from a public transport API. For example, it can use an API provided by a common transport data provider. The information obtained also includes data on congestion and delays.

[0469] The server utilizes AI algorithms to analyze acquired location and traffic information. By implementing deep learning or reinforcement learning algorithms using a generative AI model, it calculates the optimal travel route. The generated route includes detailed information such as walking routes, transfer information, and the duration of use for each mode of transport.

[0470] Next, the server sends route information to the terminal, which then guides the user through visual and audio guidance. The guidance is intuitive, using a dedicated map app and voice assistant, allowing the user to navigate accordingly.

[0471] As a concrete example, consider a scenario where a train service is suspended during the commute to school. The server retrieves the suspension information in real time and guides the user to an alternative route. For example, it might display a concise walking route to the nearest bus stop and the time of the next bus, along with voice guidance.

[0472] An example of a prompt message is: "Please tell me the best route from my current location to my destination. Public transportation is required."

[0473] This system allows users to respond flexibly to unexpected situations and travel with peace of mind.

[0474] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0475] Step 1:

[0476] The user carries the device and begins to move. The device uses its built-in GPS module to determine the user's current location. This input allows the device to obtain latitude and longitude coordinate data. This acquired location information is transmitted to the server via mobile data communication or Wi-Fi.

[0477] Step 2:

[0478] The server begins processing the location information received from the terminal. Next, the server uses a real-time traffic API to collect publicly available traffic information. Specifically, it obtains data that may affect the user's movement, such as congestion levels and delay information. This information is input to the server, creating an up-to-date traffic information database.

[0479] Step 3:

[0480] The server uses location and traffic information as input to perform data analysis using AI algorithms. Specifically, it optimizes routes using generative AI models. This is a process that generates the optimal travel route using deep learning models or reinforcement learning. As output, a travel route optimized for the user is generated.

[0481] Step 4:

[0482] The server sends the generated route information to the terminal. The terminal receives this as input and prepares directions for the user. The terminal uses a dedicated navigation app to visually display the route on a map. It also uses a voice assistant to notify the user of the next action by voice. As a result, the output provides the user with directions that are easy to understand intuitively.

[0483] Step 5:

[0484] In the event of a disruption in public transport, the server checks the latest traffic information and re-evaluates existing routes. Upon receiving the disruption information as input, the server immediately generates an alternative route. This new route information is sent to the terminal, and an emergency notification is issued to the user. As output, the user can obtain the new route guidance.

[0485] Step 6:

[0486] After the journey is complete, the user provides feedback via a terminal. The terminal collects the user's feedback data as input and sends it to the server. The server processes this feedback and uses it to continuously improve the entire system. As an output, the system's guidance functions and route generation algorithms are improved, resulting in a better user experience in the future.

[0487] (Application Example 1)

[0488] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0489] For individuals with developmental disabilities, the elderly, and others who have difficulty with mobility, there is a need to alleviate their anxieties about using public transportation to travel safely and efficiently to their destinations, and especially to enable smooth responses in the event of an emergency. In particular, since these individuals have difficulty independently resolving problems when faced with sudden changes in the environment or unpredictable troubles, the development of support systems is a challenge.

[0490] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0491] In this invention, the server includes information acquisition means for acquiring location information and environmental information, data collection means for collecting public transportation operation information, and route generation means for generating an optimal route based on the acquired location information and traffic information. This enables individuals with mobility difficulties to travel to their destinations safely when using public transportation, and also allows for a swift and appropriate response in the event of an emergency.

[0492] "Information acquisition means" refers to devices and technologies that accurately acquire the user's current location and surrounding environmental information.

[0493] "Data collection methods" refer to systems and techniques for collecting real-time information on public transportation and uploading it to a server.

[0494] "Route generation means" refers to a process or device for calculating and generating the optimal travel route based on collected location information and traffic information.

[0495] "Guidance means" refers to methods or devices that provide instructions to the user through audio or visual information based on a generated route.

[0496] "Notification means" refers to systems or technologies that automatically transmit information to pre-registered emergency contacts in the event of an emergency.

[0497] "External information processing means" refers to technologies and systems that process necessary external information for the purpose of supporting individuals with mobility difficulties.

[0498] "Environmental data" refers to information about external conditions that affect movement, such as the user's surroundings, weather, and traffic volume.

[0499] "Data processing means" refers to processes and devices used to analyze feedback information collected after movement and to improve the system.

[0500] The system for implementing this invention consists of a user-carried terminal, a server for processing data, and a network infrastructure for integrating these. First, the terminal uses a GPS module to obtain the user's current location. This location information is transmitted to the server in real time. Based on this information, the server collects public transport operation information via a real-time traffic API. Specifically, this includes weather, traffic congestion, and service delay information.

[0501] The server uses a route generation system to calculate the optimal travel route for the user based on the collected data. An AI algorithm is used for this calculation. The generated route information is sent to the terminal and presented to the user as audio and visual information by the terminal's guidance system. Text-to-speech software (e.g., Google Text-to-Speech API) is used for the audio guidance.

[0502] If the terminal detects an abnormal situation, it immediately sends information to the server and regenerates an alternative optimal route. It also sends emergency notifications to registered contacts using notification methods as needed. During this process, the user's movement history and feedback information are stored on the server and used to improve the system.

[0503] As a concrete example, if a user encounters a train cancellation on their way to a museum, the server will immediately grasp this information and provide a walking route to the nearest bus stop and a timetable for the next bus.

[0504] An example of a prompt message would be, "The train the user was planning to take to the art museum has been cancelled. Please suggest an alternative route in this situation." This allows for flexible and real-time responses through the generative AI model.

[0505] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0506] Step 1:

[0507] The terminal uses a GPS module to obtain the user's current location. It receives GPS data as input, generates location information as output, and sends it to the server. In this step, the terminal analyzes the GPS signal to obtain latitude and longitude data.

[0508] Step 2:

[0509] The server uses the received location information to collect public transport operation information using a real-time transportation API. As input, it makes requests to the API based on the location information, and as output, it obtains information such as operation status and delays. The server organizes this information and prepares it as basic data for creating optimal routes.

[0510] Step 3:

[0511] The server uses AI algorithms to generate optimal travel routes from location and traffic information. It takes well-organized location and traffic data as input, performs AI analysis, and creates optimal route data as output. This calculates efficient and safe routes for users.

[0512] Step 4:

[0513] The generated route information is sent from the server to the terminal. The terminal provides this route information to the user as audio and visual information. It receives route data from the server as input and displays navigation guides on the audio output device and display as output. Specifically, the terminal provides voice guidance using the Google Text-to-Speech API.

[0514] Step 5:

[0515] If an abnormal situation occurs, the terminal immediately sends the information to the server. The server regenerates a new, optimal route and sends it back to the terminal. In this step, the server again utilizes AI to recalculate a precise route based on the latest traffic conditions and, if necessary, sends notifications to emergency contacts via notification methods.

[0516] Step 6:

[0517] After the user finishes their journey, the terminal sends feedback information to the server. The server receives user experience evaluations as input and stores this data as output. The server uses this feedback data to improve the system. Specific actions include simple and intuitive feedback collection through the user interface.

[0518] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0519] This invention provides a navigation system incorporating an emotion engine to help users with developmental disabilities use public transportation with peace of mind. The system configuration and usage method are described below.

[0520] When a user picks up a device and begins to move, the device first determines its current location through a location information acquisition method. Then, the server obtains public transportation operation information from a transportation API to understand real-time traffic conditions. Based on this information, the optimal travel route is calculated.

[0521] The system is equipped with an emotion engine that senses the user's emotional state in real time. The device analyzes the user's facial expressions, tone of voice, and input data, and uses the emotion engine to analyze the user's emotional state. As a result, the navigation of the travel route is adjusted according to the user's stress level. For example, if anxiety is detected, the guidance is switched to a simpler and more reassuring expression.

[0522] One possible scenario is when a user feels anxious due to train delay information. Based on the latest service information received from the server, the device will guide the user to a preferred alternative route to encourage safer travel. Depending on the results of the emotion engine's analysis, voice guidance such as, "Please stay calm, I will guide you to the next bus stop," may also be provided.

[0523] Furthermore, this system uses a feedback processing mechanism to encourage friendly feedback after the move is completed, accumulating user experience points. User feedback is collected on the server and used to improve future services.

[0524] In this way, this system enhances users' sense of security when going out and traveling, thereby increasing opportunities for social participation and contributing to the improvement of users' quality of life (QOL).

[0525] The following describes the processing flow.

[0526] Step 1:

[0527] The user starts up their device and prepares to begin moving. At this stage, the emotion engine also starts up and begins measuring the user's initial emotional state.

[0528] Step 2:

[0529] The device uses GPS to obtain the user's current location. This location information is sent to the server, allowing for accurate understanding of the current environment.

[0530] Step 3:

[0531] The server retrieves real-time public transport service information via an API. This information includes data on congestion, delays, and cancellations.

[0532] Step 4:

[0533] The server uses an AI model to calculate the optimal travel route based on acquired location and operational information. It also considers the results of the emotion engine's analysis to select a stress-free route.

[0534] Step 5:

[0535] The system sends a calculated optimal travel route to the device, which then provides the user with corresponding visual navigation and voice guidance. The guidance is adjusted based on the user's emotional state.

[0536] Step 6:

[0537] While the user is on the move, the device's emotion engine continuously analyzes the user's voice and facial expressions, monitoring changes in their emotions. If an abnormal emotion is detected, the system immediately changes the guidance provided to reassure the user.

[0538] Step 7:

[0539] In the event of an emergency or traffic change, the server retrieves updated traffic information and recalculates alternative routes. This information is then sent to the terminal to guide the user to a safe travel route.

[0540] Step 8:

[0541] After the transfer is complete, the user can enter feedback and send information about their experience to their device. The feedback data is stored on the server and used for future guidance and service improvements.

[0542] (Example 2)

[0543] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0544] When users with developmental disabilities use public transportation, there is a need for a navigation system that provides optimal routes in real time according to traffic conditions, while also considering the user's emotional state and providing a sense of security. However, existing systems do not take the user's emotional state into account, and therefore fail to reduce anxiety and stress and provide a better travel experience.

[0545] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0546] In this invention, the server includes location information acquisition means for acquiring the user's own location information and surrounding situation information, information collection means for collecting real-time operation information of public transportation, route generation means for generating an optimal travel route based on the acquired location information and traffic information, and emotion analysis means for analyzing the user's emotional state and adjusting the guidance content based on that state. This makes it possible to provide customized travel guidance according to the user's real-time emotional state, giving the user a sense of security and realizing a more comfortable travel experience.

[0547] "Location information acquisition means" refers to a function for determining the user's current location, and typically involves a device that obtains latitude and longitude using technologies such as GPS.

[0548] "Information gathering means" refers to a function for acquiring data related to the operation of public transportation, and is a method for obtaining real-time operational information from transportation APIs.

[0549] A "route generation means" is a function that calculates the optimal travel route for the user based on collected location information and traffic information, and is a device that uses an algorithm to derive a route that takes time and distance into consideration.

[0550] A "guidance means" is a function that presents the generated travel route to the user visually or audibly, and is a device for appropriately conveying navigation instructions.

[0551] "Emotional analysis means" refers to a function that analyzes the user's emotional state from their facial expressions and tone of voice, and adjusts the navigation guidance based on the results obtained.

[0552] "Notification method" refers to a function that automatically sends notifications to pre-configured emergency contacts in the event of an abnormal situation.

[0553] This invention is a navigation system that enables users with developmental disabilities to use public transportation with peace of mind. The system includes means for acquiring location information, means for collecting information, means for generating routes, means for providing guidance, means for analyzing emotions, and means for providing notifications.

[0554] The device determines the user's current location using GPS-based location acquisition. The server utilizes a transportation API to collect real-time public transport operation information and calculates the optimal travel route based on that information. In route generation, an algorithm that prioritizes the shortest travel time is applied.

[0555] The device provides visual and auditory guidance along the generated travel path. An emotion analysis system recognizes the user's facial expressions and tone of voice to assess their emotional state and adjust the guidance accordingly. For example, if anxiety is detected, the guidance changes to a simpler, more reassuring style.

[0556] For example, if a user feels stressed due to congestion or delays at a train station, the system analyzes their emotional state and provides a preferred alternative route to help them relax. Based on the emotional analysis, a voice message will also play saying, "Please calm down, we will guide you to the best route to your next destination."

[0557] Furthermore, in the event of an emergency, a notification will be automatically sent to pre-configured emergency contacts via a notification system.

[0558] After the user completes their journey, the device collects feedback. The server uses this feedback to improve the service.

[0559] An example of a prompt that utilizes a generative AI model is: "Use the generative AI model to generate a natural language description of the emotion engine required for a public transportation user support system for users with developmental disabilities."

[0560] This system aims to provide navigation that is sensitive to the user's emotions, thereby creating a safer and more secure travel experience.

[0561] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0562] Step 1:

[0563] The terminal uses a location information acquisition method to obtain its current location using GPS functionality. The input is the GPS satellite signal received by the terminal, and the output is location information including latitude and longitude. This location information is used as basic data for subsequent processing.

[0564] Step 2:

[0565] The server accesses a transportation API via an information gathering mechanism to obtain real-time public transport operation information. The input is an API key that provides the latest traffic conditions, and the output is operation information data in JSON format. This information provides an important element in route generation.

[0566] Step 3:

[0567] The server calculates the optimal travel route by combining location information and operational information. The input is the user's current location and real-time operational information, and the output is optimal route data that considers the shortest travel time and the fewest transfers. The algorithm used includes Dijkstra's algorithm. This calculation is essential for formulating a convenient travel plan for the user.

[0568] Step 4:

[0569] The device uses emotion analysis to capture the user's facial expressions and voice tone through the camera and microphone. The input is this raw data, and the output is an analysis result indicating the user's emotional state. This analysis result allows for adjustments to the guidance provided.

[0570] Step 5:

[0571] The device provides guidance to the user based on the generated travel route and the results of sentiment analysis. Inputs are route data and emotional state, while outputs include visual map displays and voice guidance. For example, if user anxiety is detected, the guidance method switches to a simpler, more reassuring approach.

[0572] Step 6:

[0573] The device receives feedback from the user after they complete their journey. Input consists of user comments in text or selection format, and output is data used to improve the service. This will enhance the convenience of the service for future visits.

[0574] Step 7:

[0575] The server analyzes feedback data to improve the entire system. The input is feedback data, and the output is the extraction of areas for improvement and new service proposals. This stage allows the system to continuously evolve and improve the user experience.

[0576] (Application Example 2)

[0577] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0578] When using public transportation, both users with developmental disabilities and general users often experience anxiety and stress regarding transportation information and travel routes. In particular, a lack of consideration for emotional states prevents users from traveling with peace of mind. Furthermore, there is a need for improved services, including emergency response and feedback after use.

[0579] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0580] In this invention, the server includes an information acquisition device for acquiring location information, an information collection device for collecting transportation operation information, and an emotion analysis device for analyzing the user's emotional state. This makes it possible to provide optimal travel route guidance and a sense of security based on the user's emotional state. Furthermore, continuous improvement of the service can be achieved by incorporating feedback after use.

[0581] An "information acquisition device" is a device that acquires location information of the user's current location and surrounding environment.

[0582] An "information gathering device" is a device that collects real-time operational information of public transportation.

[0583] A "route generation device" is a device that generates the optimal travel route based on acquired location information and traffic information.

[0584] A "guidance device" is a device that presents the generated travel route to the user using both audio and visual means.

[0585] A "communication device" is a device that sends notifications to emergency contacts in the event of an abnormal situation.

[0586] An "emotion analysis device" is a device that estimates a user's emotional state in real time by analyzing their facial expressions and voice.

[0587] A "guidance adjustment device" is a device that provides customized guidance according to the user's emotional state.

[0588] A "feedback processing device" is a device that collects feedback data from users and uses it to improve services.

[0589] The system for realizing this invention is a multi-functional navigation system designed to make using public transportation safer. The system primarily consists of a user terminal and a server for processing information. The specific configuration and processing flow of the system are described below.

[0590] The server has an information acquisition device that obtains location information from the user's smartphone or other device. This device uses technologies such as GPS to determine the user's current location. The server then uses an information collection device that collects real-time operation information of public transportation to obtain the latest traffic information through a traffic API.

[0591] Next, the server uses this real-time information to generate the optimal travel route for the user using a route generation device. The software used includes a route optimization tool that employs AI algorithms. The generated route is presented to the user's terminal via a guidance device, both audibly and visually.

[0592] Furthermore, the device is equipped with an emotion analysis system that analyzes the user's facial expressions and tone of voice to estimate their emotional state in real time. This uses machine learning frameworks such as TensorFlow Lite. Based on the analyzed emotional state, the guidance adjustment system adjusts the format and content of the guidance to provide appropriate feedback to the user.

[0593] For example, if a user feels anxious about the crowded conditions at a train station, an emotion analysis device will detect this state, and the guidance adjustment device will offer reassuring guidance such as, "Why don't you stop by a nearby cafe to calm down for a bit?"

[0594] In addition, after the user's journey is complete, a feedback processing unit collects feedback from the user and uses it to improve the service. This feedback is aggregated on the server side and used to improve the service in the future.

[0595] An example of a prompt message is: "Design a traffic navigation app using an emotion engine and provide real-time guidance that responds to the user's emotions."

[0596] In this way, the system will be attentive to the user's emotions and help create a society where people can travel with a sense of security.

[0597] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0598] Step 1:

[0599] The server receives a request from the user's terminal and obtains location information via an information acquisition device. The input is the user's location data, and the output is specific latitude and longitude information. Based on this information, the server records the user's location in its database.

[0600] Step 2:

[0601] The server uses a traffic API to obtain real-time operational information from data collection devices. The input is data provided by the traffic API, and the output is real-time information showing the operational status of public transportation. This information is stored in a database and organized into a format usable for subsequent route generation.

[0602] Step 3:

[0603] The server uses acquired location and traffic information to calculate the optimal travel route using a route generation device. The input is the user's location information and real-time traffic data, and the output is the recommended travel route for the user. An AI algorithm is used in the calculation, considering the shortest travel time and routes that avoid congestion.

[0604] Step 4:

[0605] The terminal receives route data transmitted from the server and presents it to the user audibly and visually using a guidance device. The input is the route data received from the server, and the output is audio guidance and map display. The information is displayed in a way that is easy for the user to understand.

[0606] Step 5:

[0607] The device analyzes the user's facial expressions and voice in real time using an emotion analysis device. Input is sensory information from the smartphone's camera and microphone, and output is the emotional state based on the analysis. A TensorFlow Lite model is used to determine the user's emotional state.

[0608] Step 6:

[0609] The server receives the results of the emotion analysis and uses a guidance adjustment device to adjust the guidance content according to the user's emotional state. The input is the user's emotional state data, and the output is a customized guidance message. If the user is feeling anxious, the content will be changed to provide reassurance.

[0610] Step 7:

[0611] After the user reaches their destination, a feedback processing device is used to collect feedback about the service at the terminal and send it to the server. The input is feedback data from the user, and the output is feedback information stored on the server. The feedback is used to improve the service in the future.

[0612] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0613] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0614] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0615] [Fourth Embodiment]

[0616] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0617] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0618] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0619] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0620] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0621] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0622] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0623] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0624] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0625] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0626] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0627] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0628] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0629] This invention is an integrated navigation system designed to support users with developmental disabilities in safely and efficiently using public transportation. The system configuration and its usage are described below.

[0630] First, the user carries the device and begins to move. The device uses GPS to determine the user's current location and transmits this information to the server through a location information acquisition method. The server calculates the route to the user's planned destination based on public transport operation information collected from a real-time traffic API. This information includes traffic congestion and delay information.

[0631] The server uses an AI algorithm to generate the optimal travel route and sends this route information to the terminal. The terminal visually displays the route information to the user and uses voice guidance to inform the user of the next action. This makes it easier for the user to intuitively understand the instructions.

[0632] In the event of a transportation disruption, accident, or other emergency, the server will immediately generate a new alternative route and notify the user via their device. Furthermore, in an emergency, notifications will be sent to pre-registered guardians or caregivers. This function supports emergency contact methods.

[0633] As a concrete example, consider a typical commute to school. If a user's planned train is cancelled during their commute, the server immediately retrieves this information and guides the device with walking directions to the nearest bus stop and the time of the next bus. This guidance is provided through a simple map and audio guide.

[0634] After the transfer is complete, users can provide feedback on their experience through their device. This feedback is collected on the server and used to continuously improve the system. This allows for flexible responses tailored to the individual needs and challenges of each user.

[0635] In this way, this system can provide support to enable users to go out and live social lives with peace of mind, and can also contribute to building an inclusive environment for society as a whole.

[0636] The following describes the processing flow.

[0637] Step 1:

[0638] The user starts up the device and enables location services. This prepares the device to determine the user's current location.

[0639] Step 2:

[0640] The device uses GPS to obtain the user's precise location information and sends it to the server. This information serves as basic data necessary for subsequent processing.

[0641] Step 3:

[0642] The server accesses multiple transportation APIs to collect real-time information on public transport operations. It retrieves the operating status and delay information of each transportation service to understand the current traffic situation.

[0643] Step 4:

[0644] The server integrates location and traffic information and uses an AI algorithm to calculate the optimal travel route. Here, route selection is made considering both efficiency and safety.

[0645] Step 5:

[0646] The calculated travel route is sent to the terminal, which then presents the route information to the user. The display is done through a map format and voice guidance, designed to be simple and intuitive to understand.

[0647] Step 6:

[0648] While in transit, the server continuously monitors changes in traffic conditions. In the event of an emergency (such as a service cancellation or accident), a new alternative route is immediately generated.

[0649] Step 7:

[0650] If an abnormal situation occurs, the server sends a notification to the terminal, and the terminal informs the user. Additionally, notifications are sent to pre-registered emergency contacts as needed.

[0651] Step 8:

[0652] After the transfer is complete, the user provides feedback via their device. This feedback is collected by the server and used to improve future services.

[0653] (Example 1)

[0654] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0655] There is a need to provide an integrated system that alleviates the difficulties and anxieties faced by users with developmental disabilities when using public transportation, and supports safe and efficient travel. Furthermore, feedback mechanisms are necessary to continuously improve the travel experience for individual users.

[0656] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0657] In this invention, the server includes location information acquisition means for detecting and dynamically transmitting location information, information collection means for aggregating operational information in real time, and route generation means for designing optimized travel routes using AI algorithms. This enables users to travel safely and efficiently and respond quickly to abnormal situations.

[0658] A "location information acquisition means" is a technical device for accurately detecting the user's current location and transmitting it to a server.

[0659] "Information gathering means" refers to functions that efficiently acquire and aggregate real-time operational information of public transportation and make it available throughout the entire system.

[0660] "Route generation means" refers to the process of generating the optimal travel route for the user by utilizing AI algorithms based on obtained location information and traffic information.

[0661] A "guidance method" is a technology that provides users with generated travel routes visually and audibly, guiding them in an intuitively understandable way.

[0662] A "notification mechanism" is a function that provides an appropriate alternative route and quickly notifies emergency contacts of the situation when an abnormal situation occurs.

[0663] An "environmental monitoring device" is a device that collects video data of the surrounding environment and incorporates that information into the route generation process.

[0664] A "feedback processing mechanism" is a function that collects feedback data from users regarding their travel experience and uses it to improve the system.

[0665] Embodiments of this invention will now be described. This system is a navigation system designed to enable users with developmental disabilities to use public transportation safely and efficiently.

[0666] First, the device carried by the user is equipped with a GPS module, which allows the user's current location to be accurately determined. The device transmits this location information to a server. This transmission is carried out via mobile data communication or a Wi-Fi connection.

[0667] After receiving location information, the server uses data collection methods to obtain real-time public transport operation information from a public transport API. For example, it can use an API provided by a common transport data provider. The information obtained also includes data on congestion and delays.

[0668] The server utilizes AI algorithms to analyze acquired location and traffic information. By implementing deep learning or reinforcement learning algorithms using a generative AI model, it calculates the optimal travel route. The generated route includes detailed information such as walking routes, transfer information, and the duration of use for each mode of transport.

[0669] Next, the server sends route information to the terminal, which then guides the user through visual and audio guidance. The guidance is intuitive, using a dedicated map app and voice assistant, allowing the user to navigate accordingly.

[0670] As a concrete example, consider a scenario where a train service is suspended during the commute to school. The server retrieves the suspension information in real time and guides the user to an alternative route. For example, it might display a concise walking route to the nearest bus stop and the time of the next bus, along with voice guidance.

[0671] An example of a prompt message is: "Please tell me the best route from my current location to my destination. Public transportation is required."

[0672] This system allows users to respond flexibly to unexpected situations and travel with peace of mind.

[0673] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0674] Step 1:

[0675] The user carries the device and begins to move. The device uses its built-in GPS module to determine the user's current location. This input allows the device to obtain latitude and longitude coordinate data. This acquired location information is transmitted to the server via mobile data communication or Wi-Fi.

[0676] Step 2:

[0677] The server begins processing the location information received from the terminal. Next, the server uses a real-time traffic API to collect publicly available traffic information. Specifically, it obtains data that may affect the user's movement, such as congestion levels and delay information. This information is input to the server, creating an up-to-date traffic information database.

[0678] Step 3:

[0679] The server uses location and traffic information as input to perform data analysis using AI algorithms. Specifically, it optimizes routes using generative AI models. This is a process that generates the optimal travel route using deep learning models or reinforcement learning. As output, a travel route optimized for the user is generated.

[0680] Step 4:

[0681] The server sends the generated route information to the terminal. The terminal receives this as input and prepares directions for the user. The terminal uses a dedicated navigation app to visually display the route on a map. It also uses a voice assistant to notify the user of the next action by voice. As a result, the output provides the user with directions that are easy to understand intuitively.

[0682] Step 5:

[0683] In the event of a disruption in public transport, the server checks the latest traffic information and re-evaluates existing routes. Upon receiving the disruption information as input, the server immediately generates an alternative route. This new route information is sent to the terminal, and an emergency notification is issued to the user. As output, the user can obtain the new route guidance.

[0684] Step 6:

[0685] After the journey is complete, the user provides feedback via a terminal. The terminal collects the user's feedback data as input and sends it to the server. The server processes this feedback and uses it to continuously improve the entire system. As an output, the system's guidance functions and route generation algorithms are improved, resulting in a better user experience in the future.

[0686] (Application Example 1)

[0687] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0688] For individuals with developmental disabilities, the elderly, and others who have difficulty with mobility, there is a need to alleviate their anxieties about using public transportation to travel safely and efficiently to their destinations, and especially to enable smooth responses in the event of an emergency. In particular, since these individuals have difficulty independently resolving problems when faced with sudden changes in the environment or unpredictable troubles, the development of support systems is a challenge.

[0689] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0690] In this invention, the server includes information acquisition means for acquiring location information and environmental information, data collection means for collecting public transportation operation information, and route generation means for generating an optimal route based on the acquired location information and traffic information. This enables individuals with mobility difficulties to travel to their destinations safely when using public transportation, and also allows for a swift and appropriate response in the event of an emergency.

[0691] "Information acquisition means" refers to devices and technologies that accurately acquire the user's current location and surrounding environmental information.

[0692] "Data collection methods" refer to systems and techniques for collecting real-time information on public transportation and uploading it to a server.

[0693] "Route generation means" refers to a process or device for calculating and generating the optimal travel route based on collected location information and traffic information.

[0694] "Guidance means" refers to methods or devices that provide instructions to the user through audio or visual information based on a generated route.

[0695] "Notification means" refers to systems or technologies that automatically transmit information to pre-registered emergency contacts in the event of an emergency.

[0696] "External information processing means" refers to technologies and systems that process necessary external information for the purpose of supporting individuals with mobility difficulties.

[0697] "Environmental data" refers to information about external conditions that affect movement, such as the user's surroundings, weather, and traffic volume.

[0698] "Data processing means" refers to processes and devices used to analyze feedback information collected after movement and to improve the system.

[0699] The system for implementing this invention consists of a user-carried terminal, a server for processing data, and a network infrastructure for integrating these. First, the terminal uses a GPS module to obtain the user's current location. This location information is transmitted to the server in real time. Based on this information, the server collects public transport operation information via a real-time traffic API. Specifically, this includes weather, traffic congestion, and service delay information.

[0700] The server uses a route generation system to calculate the optimal travel route for the user based on the collected data. An AI algorithm is used for this calculation. The generated route information is sent to the terminal and presented to the user as audio and visual information by the terminal's guidance system. Text-to-speech software (e.g., Google Text-to-Speech API) is used for the audio guidance.

[0701] If the terminal detects an abnormal situation, it immediately sends information to the server and regenerates an alternative optimal route. It also sends emergency notifications to registered contacts using notification methods as needed. During this process, the user's movement history and feedback information are stored on the server and used to improve the system.

[0702] As a concrete example, if a user encounters a train cancellation on their way to a museum, the server will immediately grasp this information and provide a walking route to the nearest bus stop and a timetable for the next bus.

[0703] An example of a prompt message would be, "The train the user was planning to take to the art museum has been cancelled. Please suggest an alternative route in this situation." This allows for flexible and real-time responses through the generative AI model.

[0704] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0705] Step 1:

[0706] The terminal uses a GPS module to obtain the user's current location. It receives GPS data as input, generates location information as output, and sends it to the server. In this step, the terminal analyzes the GPS signal to obtain latitude and longitude data.

[0707] Step 2:

[0708] The server uses the received location information to collect public transport operation information using a real-time transportation API. As input, it makes requests to the API based on the location information, and as output, it obtains information such as operation status and delays. The server organizes this information and prepares it as basic data for creating optimal routes.

[0709] Step 3:

[0710] The server uses AI algorithms to generate optimal travel routes from location and traffic information. It takes well-organized location and traffic data as input, performs AI analysis, and creates optimal route data as output. This calculates efficient and safe routes for users.

[0711] Step 4:

[0712] The generated route information is sent from the server to the terminal. The terminal provides this route information to the user as audio and visual information. It receives route data from the server as input and displays navigation guides on the audio output device and display as output. Specifically, the terminal provides voice guidance using the Google Text-to-Speech API.

[0713] Step 5:

[0714] If an abnormal situation occurs, the terminal immediately sends the information to the server. The server regenerates a new, optimal route and sends it back to the terminal. In this step, the server again utilizes AI to recalculate a precise route based on the latest traffic conditions and, if necessary, sends notifications to emergency contacts via notification methods.

[0715] Step 6:

[0716] After the user finishes their journey, the terminal sends feedback information to the server. The server receives user experience evaluations as input and stores this data as output. The server uses this feedback data to improve the system. Specific actions include simple and intuitive feedback collection through the user interface.

[0717] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0718] This invention provides a navigation system incorporating an emotion engine to help users with developmental disabilities use public transportation with peace of mind. The system configuration and usage method are described below.

[0719] When a user picks up a device and begins to move, the device first determines its current location through a location information acquisition method. Then, the server obtains public transportation operation information from a transportation API to understand real-time traffic conditions. Based on this information, the optimal travel route is calculated.

[0720] The system is equipped with an emotion engine that senses the user's emotional state in real time. The device analyzes the user's facial expressions, tone of voice, and input data, and uses the emotion engine to analyze the user's emotional state. As a result, the navigation of the travel route is adjusted according to the user's stress level. For example, if anxiety is detected, the guidance is switched to a simpler and more reassuring expression.

[0721] One possible scenario is when a user feels anxious due to train delay information. Based on the latest service information received from the server, the device will guide the user to a preferred alternative route to encourage safer travel. Depending on the results of the emotion engine's analysis, voice guidance such as, "Please stay calm, I will guide you to the next bus stop," may also be provided.

[0722] Furthermore, this system uses a feedback processing mechanism to encourage friendly feedback after the move is completed, accumulating user experience points. User feedback is collected on the server and used to improve future services.

[0723] In this way, this system enhances users' sense of security when going out and traveling, thereby increasing opportunities for social participation and contributing to the improvement of users' quality of life (QOL).

[0724] The following describes the processing flow.

[0725] Step 1:

[0726] The user starts up their device and prepares to begin moving. At this stage, the emotion engine also starts up and begins measuring the user's initial emotional state.

[0727] Step 2:

[0728] The device uses GPS to obtain the user's current location. This location information is sent to the server, allowing for accurate understanding of the current environment.

[0729] Step 3:

[0730] The server retrieves real-time public transport service information via an API. This information includes data on congestion, delays, and cancellations.

[0731] Step 4:

[0732] The server uses an AI model to calculate the optimal travel route based on acquired location and operational information. It also considers the results of the emotion engine's analysis to select a stress-free route.

[0733] Step 5:

[0734] The system sends a calculated optimal travel route to the device, which then provides the user with corresponding visual navigation and voice guidance. The guidance is adjusted based on the user's emotional state.

[0735] Step 6:

[0736] While the user is on the move, the device's emotion engine continuously analyzes the user's voice and facial expressions, monitoring changes in their emotions. If an abnormal emotion is detected, the system immediately changes the guidance provided to reassure the user.

[0737] Step 7:

[0738] In the event of an emergency or traffic change, the server retrieves updated traffic information and recalculates alternative routes. This information is then sent to the terminal to guide the user to a safe travel route.

[0739] Step 8:

[0740] After the transfer is complete, the user can enter feedback and send information about their experience to their device. The feedback data is stored on the server and used for future guidance and service improvements.

[0741] (Example 2)

[0742] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0743] When users with developmental disabilities use public transportation, there is a need for a navigation system that provides optimal routes in real time according to traffic conditions, while also considering the user's emotional state and providing a sense of security. However, existing systems do not take the user's emotional state into account, and therefore fail to reduce anxiety and stress and provide a better travel experience.

[0744] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0745] In this invention, the server includes location information acquisition means for acquiring the user's own location information and surrounding situation information, information collection means for collecting real-time operation information of public transportation, route generation means for generating an optimal travel route based on the acquired location information and traffic information, and emotion analysis means for analyzing the user's emotional state and adjusting the guidance content based on that state. This makes it possible to provide customized travel guidance according to the user's real-time emotional state, giving the user a sense of security and realizing a more comfortable travel experience.

[0746] "Location information acquisition means" refers to a function for determining the user's current location, and typically involves a device that obtains latitude and longitude using technologies such as GPS.

[0747] "Information gathering means" refers to a function for acquiring data related to the operation of public transportation, and is a method for obtaining real-time operational information from transportation APIs.

[0748] A "route generation means" is a function that calculates the optimal travel route for the user based on collected location information and traffic information, and is a device that uses an algorithm to derive a route that takes time and distance into consideration.

[0749] A "guidance means" is a function that presents the generated travel route to the user visually or audibly, and is a device for appropriately conveying navigation instructions.

[0750] "Emotional analysis means" refers to a function that analyzes the user's emotional state from their facial expressions and tone of voice, and adjusts the navigation guidance based on the results obtained.

[0751] "Notification method" refers to a function that automatically sends notifications to pre-configured emergency contacts in the event of an abnormal situation.

[0752] This invention is a navigation system that enables users with developmental disabilities to use public transportation with peace of mind. The system includes means for acquiring location information, means for collecting information, means for generating routes, means for providing guidance, means for analyzing emotions, and means for providing notifications.

[0753] The device determines the user's current location using GPS-based location acquisition. The server utilizes a transportation API to collect real-time public transport operation information and calculates the optimal travel route based on that information. In route generation, an algorithm that prioritizes the shortest travel time is applied.

[0754] The device provides visual and auditory guidance along the generated travel path. An emotion analysis system recognizes the user's facial expressions and tone of voice to assess their emotional state and adjust the guidance accordingly. For example, if anxiety is detected, the guidance changes to a simpler, more reassuring style.

[0755] For example, if a user feels stressed due to congestion or delays at a train station, the system analyzes their emotional state and provides a preferred alternative route to help them relax. Based on the emotional analysis, a voice message will also play saying, "Please calm down, we will guide you to the best route to your next destination."

[0756] Furthermore, in the event of an emergency, a notification will be automatically sent to pre-configured emergency contacts via a notification system.

[0757] After the user completes their journey, the device collects feedback. The server uses this feedback to improve the service.

[0758] An example of a prompt that utilizes a generative AI model is: "Use the generative AI model to generate a natural language description of the emotion engine required for a public transportation user support system for users with developmental disabilities."

[0759] This system aims to provide navigation that is sensitive to the user's emotions, thereby creating a safer and more secure travel experience.

[0760] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0761] Step 1:

[0762] The terminal uses a location information acquisition method to obtain its current location using GPS functionality. The input is the GPS satellite signal received by the terminal, and the output is location information including latitude and longitude. This location information is used as basic data for subsequent processing.

[0763] Step 2:

[0764] The server accesses a transportation API via an information gathering mechanism to obtain real-time public transport operation information. The input is an API key that provides the latest traffic conditions, and the output is operation information data in JSON format. This information provides an important element in route generation.

[0765] Step 3:

[0766] The server calculates the optimal travel route by combining location information and operational information. The input is the user's current location and real-time operational information, and the output is optimal route data that considers the shortest travel time and the fewest transfers. The algorithm used includes Dijkstra's algorithm. This calculation is essential for formulating a convenient travel plan for the user.

[0767] Step 4:

[0768] The device uses emotion analysis to capture the user's facial expressions and voice tone through the camera and microphone. The input is this raw data, and the output is an analysis result indicating the user's emotional state. This analysis result allows for adjustments to the guidance provided.

[0769] Step 5:

[0770] The device provides guidance to the user based on the generated travel route and the results of sentiment analysis. Inputs are route data and emotional state, while outputs include visual map displays and voice guidance. For example, if user anxiety is detected, the guidance method switches to a simpler, more reassuring approach.

[0771] Step 6:

[0772] The device receives feedback from the user after they complete their journey. Input consists of user comments in text or selection format, and output is data used to improve the service. This will enhance the convenience of the service for future visits.

[0773] Step 7:

[0774] The server analyzes feedback data to improve the entire system. The input is feedback data, and the output is the extraction of areas for improvement and new service proposals. This stage allows the system to continuously evolve and improve the user experience.

[0775] (Application Example 2)

[0776] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0777] When using public transportation, both users with developmental disabilities and general users often experience anxiety and stress regarding transportation information and travel routes. In particular, a lack of consideration for emotional states prevents users from traveling with peace of mind. Furthermore, there is a need for improved services, including emergency response and feedback after use.

[0778] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0779] In this invention, the server includes an information acquisition device for acquiring location information, an information collection device for collecting transportation operation information, and an emotion analysis device for analyzing the user's emotional state. This makes it possible to provide optimal travel route guidance and a sense of security based on the user's emotional state. Furthermore, continuous improvement of the service can be achieved by incorporating feedback after use.

[0780] An "information acquisition device" is a device that acquires location information of the user's current location and surrounding environment.

[0781] An "information gathering device" is a device that collects real-time operational information of public transportation.

[0782] A "route generation device" is a device that generates the optimal travel route based on acquired location information and traffic information.

[0783] A "guidance device" is a device that presents the generated travel route to the user using both audio and visual means.

[0784] A "communication device" is a device that sends notifications to emergency contacts in the event of an abnormal situation.

[0785] An "emotion analysis device" is a device that estimates a user's emotional state in real time by analyzing their facial expressions and voice.

[0786] A "guidance adjustment device" is a device that provides customized guidance according to the user's emotional state.

[0787] A "feedback processing device" is a device that collects feedback data from users and uses it to improve services.

[0788] The system for realizing this invention is a multi-functional navigation system designed to make using public transportation safer. The system primarily consists of a user terminal and a server for processing information. The specific configuration and processing flow of the system are described below.

[0789] The server has an information acquisition device that obtains location information from the user's smartphone or other device. This device uses technologies such as GPS to determine the user's current location. The server then uses an information collection device that collects real-time operation information of public transportation to obtain the latest traffic information through a traffic API.

[0790] Next, the server uses this real-time information to generate the optimal travel route for the user using a route generation device. The software used includes a route optimization tool that employs AI algorithms. The generated route is presented to the user's terminal via a guidance device, both audibly and visually.

[0791] Furthermore, the device is equipped with an emotion analysis system that analyzes the user's facial expressions and tone of voice to estimate their emotional state in real time. This uses machine learning frameworks such as TensorFlow Lite. Based on the analyzed emotional state, the guidance adjustment system adjusts the format and content of the guidance to provide appropriate feedback to the user.

[0792] For example, if a user feels anxious about the crowded conditions at a train station, an emotion analysis device will detect this state, and the guidance adjustment device will offer reassuring guidance such as, "Why don't you stop by a nearby cafe to calm down for a bit?"

[0793] In addition, after the user's journey is complete, a feedback processing unit collects feedback from the user and uses it to improve the service. This feedback is aggregated on the server side and used to improve the service in the future.

[0794] An example of a prompt message is: "Design a traffic navigation app using an emotion engine and provide real-time guidance that responds to the user's emotions."

[0795] In this way, the system will be attentive to the user's emotions and help create a society where people can travel with a sense of security.

[0796] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0797] Step 1:

[0798] The server receives a request from the user's terminal and obtains location information via an information acquisition device. The input is the user's location data, and the output is specific latitude and longitude information. Based on this information, the server records the user's location in its database.

[0799] Step 2:

[0800] The server uses a traffic API to obtain real-time operational information from data collection devices. The input is data provided by the traffic API, and the output is real-time information showing the operational status of public transportation. This information is stored in a database and organized into a format usable for subsequent route generation.

[0801] Step 3:

[0802] The server uses acquired location and traffic information to calculate the optimal travel route using a route generation device. The input is the user's location information and real-time traffic data, and the output is the recommended travel route for the user. An AI algorithm is used in the calculation, considering the shortest travel time and routes that avoid congestion.

[0803] Step 4:

[0804] The terminal receives route data transmitted from the server and presents it to the user audibly and visually using a guidance device. The input is the route data received from the server, and the output is audio guidance and map display. The information is displayed in a way that is easy for the user to understand.

[0805] Step 5:

[0806] The device analyzes the user's facial expressions and voice in real time using an emotion analysis device. Input is sensory information from the smartphone's camera and microphone, and output is the emotional state based on the analysis. A TensorFlow Lite model is used to determine the user's emotional state.

[0807] Step 6:

[0808] The server receives the results of the emotion analysis and uses a guidance adjustment device to adjust the guidance content according to the user's emotional state. The input is the user's emotional state data, and the output is a customized guidance message. If the user is feeling anxious, the content will be changed to provide reassurance.

[0809] Step 7:

[0810] After the user reaches their destination, a feedback processing device is used to collect feedback about the service at the terminal and send it to the server. The input is feedback data from the user, and the output is feedback information stored on the server. The feedback is used to improve the service in the future.

[0811] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0812] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0813] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0814] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0815] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0816] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0817] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0818] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0819] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0820] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0821] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0822] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0823] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0824] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0825] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0826] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0827] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0828] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0829] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0830] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0831] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0832] The following is further disclosed regarding the embodiments described above.

[0833] (Claim 1)

[0834] A location information acquisition means that obtains the user's own location information and surrounding environment information,

[0835] A means of collecting information to collect real-time operational information of public transportation,

[0836] A route generation means that generates the optimal travel route based on acquired location information and traffic information,

[0837] A guidance system that presents the generated travel route to the user via audio and visual means,

[0838] A notification method that sends a notification to an emergency contact in the event of an abnormal situation,

[0839] A system that includes this.

[0840] (Claim 2)

[0841] The system according to claim 1, which acquires environmental video data from a public monitoring device and takes that environmental video data into consideration when generating a travel route using the route generation means described above.

[0842] (Claim 3)

[0843] The system according to claim 1, which includes a feedback processing means for collecting user feedback data after use and reflecting it in improvements to route generation means and guidance means.

[0844] "Example 1"

[0845] (Claim 1)

[0846] A means for acquiring location information to detect and dynamically transmit location information,

[0847] A means of collecting information to aggregate operational information in real time,

[0848] A path generation means that uses AI algorithms to design optimized travel paths,

[0849] Visual and audio guidance means for providing users with a designed travel path,

[0850] A notification system that provides alternative routes in the event of an emergency and promptly notifies emergency contacts,

[0851] A system that includes this.

[0852] (Claim 2)

[0853] The system according to claim 1, which utilizes data from an environmental monitoring device and reflects it in the process of determining the route generation means.

[0854] (Claim 3)

[0855] The system according to claim 1, comprising processing means for collecting user experience-based feedback information and using it to adapt and improve route generation and guidance means.

[0856] "Application Example 1"

[0857] (Claim 1)

[0858] Information acquisition means for obtaining location information and environmental information from the user,

[0859] A data collection method for collecting information on the operation of public transportation,

[0860] A route generation means that generates the optimal route based on acquired location information and traffic information,

[0861] A guidance means that presents the generated route to the user via audio and visual means,

[0862] A notification method that sends a notification to an emergency contact in the event of an abnormal situation,

[0863] External information processing means intended to support individuals with mobility difficulties,

[0864] A system that includes this.

[0865] (Claim 2)

[0866] The system according to claim 1, which acquires environmental data and takes that data into consideration when generating a route using a route generation means.

[0867] (Claim 3)

[0868] The system according to claim 1, comprising data processing means for collecting feedback information after movement and reflecting it in improvements to route generation means and guidance means.

[0869] "Example 2 of combining an emotion engine"

[0870] (Claim 1)

[0871] A location information acquisition means that obtains the user's own location information and surrounding situation information,

[0872] Information gathering means for collecting real-time operational information of public transportation,

[0873] A route generation means that generates the optimal travel route based on acquired location information and traffic information,

[0874] A guidance system that presents the generated travel route to the user via audio and visual means,

[0875] An emotion analysis means that analyzes the user's emotional state and adjusts the guidance content based on that state,

[0876] A notification method that sends a notification to an emergency contact in the event of an abnormal situation,

[0877] A system that includes this.

[0878] (Claim 2)

[0879] The system according to claim 1, which acquires environmental video data from a public monitoring device and takes that environmental video data into consideration when generating a travel route using the route generation means described above.

[0880] (Claim 3)

[0881] The system according to claim 1, which includes a feedback processing means for collecting user feedback data after use and reflecting it in improvements to route generation means, guidance means, and sentiment analysis means.

[0882] "Application example 2 when combining with an emotional engine"

[0883] (Claim 1)

[0884] An information acquisition device that acquires location information,

[0885] An information gathering device that collects information on the operation of public transportation,

[0886] A route generation device that generates a travel route based on acquired information,

[0887] A guidance device that presents the generated route in audio and visual form,

[0888] A communication device that sends notifications in emergencies,

[0889] An emotion analysis device that analyzes the emotional state of the user,

[0890] A guidance adjustment device that provides guidance according to emotional state,

[0891] A system that includes this.

[0892] (Claim 2)

[0893] The system according to claim 1, which acquires environmental video data and takes the environmental video data into consideration when generating a travel path using the above-mentioned path generation device.

[0894] (Claim 3)

[0895] The system according to claim 1, comprising a feedback processing device that collects feedback data after use and reflects it in improvements to the route generation device and guidance device. [Explanation of symbols]

[0896] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Information acquisition means for obtaining location information and environmental information from the user, A data collection method for collecting information on the operation of public transportation, A route generation means that generates the optimal route based on acquired location information and traffic information, A guidance means that presents the generated route to the user via audio and visual means, A notification method that sends a notification to an emergency contact in the event of an abnormal situation, External information processing means intended to support individuals with mobility difficulties, A system that includes this.

2. The system according to claim 1, which acquires environmental data and takes that data into consideration when generating a route using a route generation means.

3. The system according to claim 1, comprising a data processing means for collecting feedback information after movement and reflecting it in improvements to route generation means and guidance means.

Images