system

Abstract

We provide the system. [Solution] A means for collecting location information of barrier-free facilities from a database containing geographic information, A means of using artificial intelligence technology to analyze collected location information and identify the locations of elevators and multipurpose restrooms, A means of providing accessibility information to users based on a designated route, A means of updating information in real time and supporting users' movement, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] With the aging of society, the number of people with mobility restrictions is increasing. However, in conventional map information providing services, the location information of barrier-free facilities is not sufficiently provided, so it has been difficult to move around with confidence. In particular, in complex facilities such as inside a station, there is a problem that it is difficult to quickly and accurately grasp the locations of elevators and multi-purpose toilets. As a result, the inconvenience and stress during movement increase, and it is important to solve the problem that a barrier-free movement environment is not sufficiently established.

Means for Solving the Problems

[0005] This invention provides a means for identifying and providing users with the locations of elevators and multipurpose restrooms by collecting location information of barrier-free facilities from a database containing geographic information and analyzing that information using artificial intelligence technology. Furthermore, it realizes a system that supports safe and efficient movement for users by presenting barrier-free information to users based on a specified route and updating the information in real time. This system provides information in a visually easy-to-understand format and presents multiple route options, enabling users to select the optimal route based on comfort during their journey.

[0006] "Geographic information" refers to various types of data related to geographical location, and is the information used in maps and location-based services.

[0007] A "database" is a collection of data stored in a specific format for the purpose of efficiently managing and operating information.

[0008] "Barrier-free facilities" refer to facilities designed to be easily accessible to people with physical limitations, and include elevators, multi-purpose restrooms, and other such facilities.

[0009] "Artificial intelligence technology" refers to technologies that enable computers to mimic human intellectual activity and perform learning and reasoning.

[0010] "Analyzing" is the act of systematically interpreting data and information to derive understanding and meaning from them.

[0011] An "elevator" is a lifting device installed to assist with vertical movement within a building.

[0012] A "multipurpose toilet" refers to a toilet equipped with ample space and special facilities to meet the diverse needs of various users.

[0013] A "route" refers to the path taken to travel from one point to a destination.

[0014] "Real-time" indicates that processing and reaction are performed at the moment when data or information is generated.

[0015] "Visual" refers to the presentation of information in a form that can be recognized by seeing with the eyes.

[0016] "Optimal route" refers to the path that is considered the most efficient or desirable for specific conditions or purposes.

Brief Explanation of Drawings

[0017] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [[ID=3​​​​​​​​​​​​​​​​​ [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of a plurality of arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] This invention is a system that provides users with mobility limitations with location information of barrier-free facilities based on geographic information. Specifically, a server collects geographic information and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact location of elevators and multi-purpose restrooms on a map.

[0039] The terminal retrieves updated accessibility information from the server and presents it to the user visually. Once the user enters their departure and destination points, the terminal suggests the most suitable accessibility route based on the specified path. Furthermore, the terminal updates information in real time, keeping the user informed of the latest conditions during their journey. This allows users to constantly monitor elevator availability and the availability of accessible restrooms, enabling safe and comfortable travel.

[0040] As a concrete example, consider the case of a wheelchair user using a train station in a major metropolitan area. The user can select their desired station on a terminal and check the location of barrier-free facilities. The terminal displays elevator icons on a map and provides associated route information. Furthermore, as the user progresses, the terminal issues warnings and alerts, such as notifying the user of information about temporary facility stoppages or congestion. In this way, the present invention provides specific technical means to support the smooth movement of users.

[0041] The following describes the processing flow.

[0042] Step 1:

[0043] The server collects accessibility-related information, such as elevators and multi-purpose restrooms, from APIs of public institutions and related facilities. The server stores this information in a database and performs periodic updates.

[0044] Step 2:

[0045] The server uses artificial intelligence technology to analyze the information stored in the database and identify geographical information and the locations of barrier-free facilities. The identified information is stored in an optimized format to enable efficient access.

[0046] Step 3:

[0047] When a user launches an app, the device retrieves the latest accessibility information from the server. The device caches this information, making it immediately available for use.

[0048] Step 4:

[0049] The user enters their departure and destination points into the terminal's interface. Based on this information, the terminal calculates the optimal route, taking into account barrier-free facilities.

[0050] Step 5:

[0051] The terminal displays the calculated route on a map and visually indicates icons for elevators and accessible restrooms. Users can then review and select their route based on this information.

[0052] Step 6:

[0053] While in transit, the device periodically retrieves real-time information from the server and notifies the user of any changes or alerts. This allows the user to change their route as needed or continue traveling safely.

[0054] Step 7:

[0055] When a user completes a journey, the terminal collects data such as the route taken and the time spent, and sends this feedback to the server for future optimization. This allows the system to be continuously improved.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] Providing accurate and timely accessibility information based on geographical data remains a complex challenge for those with mobility limitations. In particular, the lack of systems that can respond immediately to real-time information changes is a significant problem. Therefore, there is a need to provide necessary information in a timely and visually easy-to-understand manner, supporting the development of safe and efficient routes.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for collecting location information of barrier-free facilities from an information aggregate including geographic information, means for analyzing the collected location information using computer learning technology to identify the locations of elevators and multipurpose facilities, and means for updating information on unavailability and changes in congestion in real time and adapting the information to the user's movements. As a result, users can always obtain the latest barrier-free information and enjoy an environment in which they can move around with peace of mind.

[0061] "Geographic information" refers to various types of data related to a specific location, including information such as location, coordinates, and the arrangement of facilities.

[0062] "Information aggregation" refers to a database or similar system that centrally collects data and structures it into a usable format.

[0063] "Barrier-free facilities" refer to facilities designed to be easily used by people with mobility limitations, and specifically include elevators and multi-purpose restrooms.

[0064] "Computer learning technology" refers to computer technology that analyzes data, identifies patterns, and automatically makes predictions and decisions.

[0065] "Lifting equipment" refers to mechanical equipment used to move between different floors within a building, and generally refers to elevators.

[0066] A "multipurpose facility" refers to a facility designed to be suitable for various uses, and specifically includes facilities such as wheelchair-accessible restrooms.

[0067] "User" refers to an individual or their supporter who uses this system to obtain and move information.

[0068] "Real-time updates" refers to a process that ensures information is updated instantly and always remains up-to-date.

[0069] This invention is a system that provides users with mobility limitations with location information of barrier-free facilities based on geographical data. Three entities—a server, a terminal, and a user—work together to propose the optimal travel route to the user through the processes of information collection, analysis, and provision.

[0070] The server collects information from public institutions and related facilities, gathering geographical and accessibility information. Web crawling technology and APIs can be used for this process. The collected data is stored using a relational database management system for later analysis. MySQL® and PostgreSQL are suitable databases for this purpose.

[0071] Next, the server uses computer learning techniques to analyze the data. It utilizes machine learning libraries such as TENSORFLOW® to accurately pinpoint the locations of elevators and multi-purpose facilities. Through processes such as data cleansing, feature extraction, and predictive model execution, it generates appropriate location information.

[0072] The terminal receives the latest data from the server and provides information to the user visually. To this end, it utilizes the Google® Maps API to display information on a map. The locations of elevators and multi-purpose facilities are shown as icons, and route information is provided according to the user's needs. The terminal also updates new information obtained in real time according to the user's actions and provides appropriate notifications.

[0073] The user enters their departure and destination points into the terminal, and the system generates the optimal barrier-free route based on that information. This ensures that users can always travel based on the latest information. For example, by entering a prompt such as, "Tell me the barrier-free route from Tokyo Station to Shibuya Station," the terminal will provide the most suitable barrier-free information for the specified route.

[0074] This system aims to provide a technical framework that allows users to move around public spaces safely and to reduce barriers to access.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] The server collects geographical and accessibility information from public institutions and related facilities. The input for this step is data obtained from each facility's official website or public API. For specific data processing, web crawling technology is used to extract text data and format it into a specific format. This results in output that can be stored in a database.

[0078] Step 2:

[0079] The server stores the collected information in a database. The input is the accessibility information formatted in Step 1. Here, a relational database management system is used to store the data in the appropriate tables. The output is structured data that can be used for searching and analysis.

[0080] Step 3:

[0081] The server analyzes information in the database using computer learning techniques. The input is accessibility location information stored in the database. The server trains a machine learning model and uses it to pinpoint the exact locations of elevators and multi-purpose facilities. Specific operations include data cleansing and feature extraction. The output is a dataset with location information and prediction confidence scores added.

[0082] Step 4:

[0083] The terminal retrieves parsed information from the server and prepares it for the user. The input is location data obtained in step 3. The terminal uses the Google Maps API to visually display barrier-free facilities on a map. Specifically, it uses icons and labels to indicate the location of facilities. The output is visual information on an interface that is easy for the user to view.

[0084] Step 5:

[0085] The user enters their origin and destination into the terminal and receives suggestions for the most suitable barrier-free route. The input consists of geographical selection information provided by the user. The terminal uses Dijkstra's algorithm or the A algorithm to search for the specified route and calculate the optimal route. The output is the recommended route presented to the user and its detailed information.

[0086] Step 6:

[0087] The device updates information based on new data obtained in real time. Inputs include push notifications from the server and feed information from external data sources. Specific actions in this step include retrieving data via API and updating local data. As output, the user is provided with alerts regarding the latest congestion status and facility availability.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] For users with mobility limitations, choosing safe and comfortable routes in public spaces and over long distances can be challenging. In particular, a lack of information on accessible barrier-free facilities often leads to inconvenience and anxiety during travel. Furthermore, the lack of real-time optimization of route selection results in wasted time and effort. Solutions to these problems are urgently needed.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for collecting location information of barrier-free facilities from information storage means including geographic information; means for analyzing the collected location information using machine learning technology to identify the locations of vertical movement devices and multi-purpose spaces; means for providing barrier-free information to users based on a specified route in an automatically controlled moving medium; and means for optimizing the travel route in cooperation with the automatically controlled moving medium. As a result, users with mobility limitations can reach their destination safely and efficiently using a barrier-free route optimized in real time.

[0093] A "means of storing information including geographic information" refers to a database system that stores spatial information such as map data and geographic coordinates in digital format and makes it accessible as needed.

[0094] A "barrier-free facility" is a building or facility designed to be safely used by people with mobility limitations, and includes facilities such as elevators and multi-purpose restrooms.

[0095] "Machine learning technology" is an information processing technology that analyzes large amounts of data, learns patterns and regularities from it, and automatically makes optimal decisions and predictions.

[0096] A "vertical movement device" is a device that enables vertical movement both inside and outside a building, and typically refers to elevators or stairlifts.

[0097] A "multipurpose space" is a space equipped with facilities that can accommodate various purposes, and a multipurpose restroom is a typical example of this.

[0098] An "automatically controlled mobile medium" refers to a vehicle or device that can move autonomously or semi-autonomously, such as an autonomous vehicle.

[0099] A "real-time optimized barrier-free route" refers to the safest and most efficient route provided to users based on the latest information at all times during their journey.

[0100] This invention provides a barrier-free travel route that is useful for users with mobility limitations when using an automated mobile medium. The system consists of a server, a terminal, and a user.

[0101] The server first uses "information storage means including geographic information" to collect location information of relevant barrier-free facilities. This includes digital map data and geographic coordinates. Next, it analyzes the collected information using "machine learning technology" to identify the inherent location and usage status of the facilities. As a result, the necessary information is provided in a constantly updated form on the automatically controlled mobile medium.

[0102] The terminal visualizes this information as a "real-time optimized barrier-free route" and provides it to the user. In this process, the terminal functions as a user interface, and the information is displayed in an easy-to-understand manner by the information processing device. Through the terminal, users can understand the distance to their destination and the current traffic conditions in real time.

[0103] For example, when a wheelchair user travels in a large city using an autonomous vehicle, this system visually suggests an appropriate route based on the user's current location and destination. Furthermore, it instantly updates and displays information regarding elevator availability and any necessary route changes. The technology used in this system includes the Google Maps API and systems supporting machine learning.

[0104] Examples of prompt statements are as follows:

[0105] "You are a designer of an application that assists users with mobility limitations. Advise us on how to provide real-time information on accessible routes and facilities to ensure safe and comfortable travel for users of autonomous vehicles to their destinations."

[0106] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0107] Step 1:

[0108] The server collects location information for barrier-free facilities from information storage systems that include geographic information. It uses stored map data and lists of barrier-free facilities as input. Using this information, it accesses a database and extracts location information for vertical mobility devices and multi-purpose spaces. The output is structured location data for barrier-free facilities.

[0109] Step 2:

[0110] The server analyzes location information obtained using machine learning techniques. Inputs include location data obtained in Step 1 and historical usage data. A machine learning algorithm is executed to predict elevator operation status and the availability of multi-purpose spaces. The output is a real-time updated list of available barrier-free facilities.

[0111] Step 3:

[0112] The server generates optimal barrier-free route information based on the specified route. Input includes the user's origin and destination, and real-time traffic data. A route calculation algorithm is applied to this information to identify the optimal route. The output is detailed barrier-free route information provided to the user.

[0113] Step 4:

[0114] The terminal visualizes the barrier-free route information received from the server and displays it to the user. The input includes the route information identified in step 3. A map application is used to visually display the route clearly on the screen. The output is a user-friendly map display.

[0115] Step 5:

[0116] The user selects an accessible route via their device and begins their journey. Inputs include the displayed route and real-time facility usage information. During the journey, the user follows the planned route guided by the application. The output is a smooth route that ensures the user safely reaches their destination.

[0117] Step 6:

[0118] The device updates information in real time while the user is on the move and notifies them. Inputs include changes in traffic conditions and sudden changes in facility usage. It acquires new information and updates the user's route selection as needed. As a result, the user can continue to travel optimally based on the latest information. Outputs are the updated, most recent travel route and warning information.

[0119] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0120] This invention combines a system that provides users with mobility limitations with location information of barrier-free facilities based on geographical data, and an emotion engine that recognizes the user's emotions. Specifically, a server collects geographical and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact locations of elevators and accessible restrooms on a map. This helps in planning travel routes.

[0121] The terminal retrieves updated accessibility information from the server and presents it to the user in a visually easy-to-understand format. Furthermore, it uses an emotion engine to recognize the user's emotional state in real time. When the user enters their starting point and destination, the terminal suggests an optimized accessibility route based on the specified route, taking into account the user's stress level and emotional state.

[0122] As a concrete example, consider the case of an elderly user using a train station in a large city they are unfamiliar with. The user selects their desired station and route on the terminal, and the emotion engine analyzes the user's emotional state from their facial expressions and voice input. For example, if the user is feeling anxious, the emotion engine adjusts the route and guidance to make it more reassuring. Furthermore, if the emotional state changes during the journey, the terminal detects the change and communicates with the server to update the information. As a result, the user receives detailed guidance tailored to their individual needs, enabling a comfortable journey. This system aims to reduce stress in real time, improve the user experience, and provide technology that promotes social harmony.

[0123] The following describes the processing flow.

[0124] Step 1:

[0125] The server collects geographical information and information on barrier-free facilities using APIs provided by public institutions and facilities. The collected information is stored in a database, and detailed mapping data, including location information, is compiled.

[0126] Step 2:

[0127] The server uses artificial intelligence technology to analyze information stored in the database and identify the locations of elevators, ramps, and accessible restrooms within train stations and urban areas. This process accurately maps the relationship between geographical information and barrier-free facilities.

[0128] Step 3:

[0129] When the app is launched, the device retrieves the latest accessibility information from the server and caches it locally. This prepares the device to display information in real time.

[0130] Step 4:

[0131] The user enters their starting point and destination into the app. Based on this information, the device calculates the optimal travel route and displays it on a map in an easy-to-understand visual format.

[0132] Step 5:

[0133] The emotion engine built into the device uses the user's camera footage and voice input to analyze the user's emotional state in real time. This process measures the user's stress and anxiety levels.

[0134] Step 6:

[0135] The device customizes the suggested routes and information it presents based on the user's emotional state. For example, if the user is feeling anxious, the device prioritizes presenting simple and intuitive routes and strives to provide clear explanations.

[0136] Step 7:

[0137] While the user is on the move, the emotion engine continuously monitors the user's state, and if it detects a change, the device will modify the guidance content as needed or send a request to the server to obtain the latest information.

[0138] Step 8:

[0139] Upon reaching the destination, the device evaluates the user's travel experience and sends the accumulated data as feedback to the server. The server analyzes this data to help improve the algorithm.

[0140] (Example 2)

[0141] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0142] It is necessary to provide safe and comfortable means of transportation for users with mobility limitations, such as people with disabilities and the elderly. However, current systems have limited access to barrier-free information and cannot suggest routes that take into account the emotional state of users, which can potentially cause stress. To solve this problem, a system is needed that updates information in real time and provides guidance tailored to the individual needs of users.

[0143] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0144] In this invention, the server includes means for acquiring location data of facilities for people with disabilities from a storage device containing geographic information, means for analyzing the acquired location data using machine learning techniques to identify the locations of elevators and multi-purpose equipment within the vehicle, and means for providing information for people with disabilities to the user based on the selected route. This makes it possible to provide a safe and comfortable travel experience by recognizing the user's emotional state and proposing optimized route suggestions in real time.

[0145] A "storage device containing geographic information" is a device that stores location data and related information about multiple facilities and places, and is used by users to obtain the desired location information.

[0146] "Facilities for people with disabilities" refers to equipment and places that have been designed or modified to be easily accessible to people with disabilities or the elderly, and includes, for example, barrier-free elevators and multi-purpose restrooms.

[0147] "Location data" refers to data that includes geographical coordinate information for a specific place or facility, and is used to identify its location on a map.

[0148] "Machine learning techniques" are technologies that automate specific tasks or decisions by having computers analyze large amounts of data and learn patterns and regularities.

[0149] "Vehicle-based lifting device" refers to a vertical movement device used to move between floors within a building, and primarily refers to an elevator.

[0150] "Multipurpose facilities" refer to facilities with a combination of functions that can accommodate various uses, and specifically include facilities such as restrooms that are accessible to wheelchair users.

[0151] "User" refers to an individual who uses this system or service, and primarily includes people with disabilities and the elderly.

[0152] "Recognizing emotional states" refers to identifying and understanding the user's emotions from information such as facial expressions and voice, and is a function that is realized through technologies such as emotion engines.

[0153] "To suggest" refers to presenting a user with specific options or actions, and is an act of helping the user make the best decision.

[0154] In an embodiment for carrying out this invention, the system is configured as follows.

[0155] Server Functions

[0156] The server collects geographic information from public institutions and related facilities and stores it primarily in storage devices. This geographic information includes location data for facilities accessible to people with disabilities. To obtain this data, the server uses technology to download information via API communication. For example, the server receives data in JSON format and stores it in a database. The stored data is analyzed using machine learning techniques to identify the locations of specific facilities accessible to people with disabilities, such as elevators and accessible restrooms. Programming languages ​​such as Python and R, and their data analysis libraries, are used.

[0157] Terminal role

[0158] The terminal retrieves the latest location information from the server and provides it to the user. To this end, the terminal has the functionality to display a map application, providing information that is visually easy to understand and accessible for people with disabilities. Users can input their current location and destination using a smartphone or dedicated mobile device and receive the results. Furthermore, the terminal is equipped with an emotion engine that analyzes the user's facial expressions and voice in real time through the camera and microphone to recognize their emotional state. It uses a machine learning model to determine emotions and suggest the optimal route. It also makes adjustments to reduce the user's stress based on their emotional state, such as anxiety or reassurance.

[0159] Specific example

[0160] For example, when an elderly user visits an unfamiliar city, the user enters their starting point and destination into the device. The device reads the user's facial expressions through its built-in camera, and if the emotion engine detects that the user is anxious, it suggests a route that responds to that anxiety. Specifically, it can recommend a route that prioritizes the use of elevators over escalators.

[0161] Example of a prompt

[0162] "Please propose a reassuring guidance method to provide when users feel anxious."

[0163] This configuration allows the system to understand user needs in real time and provide a comfortable and safe travel experience.

[0164] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0165] Step 1:

[0166] The server retrieves location data for facilities accessible to people with disabilities from a storage device containing geographic information. The input is raw data obtained via API communication, and the output is structured location information stored in a database. At this stage, the server periodically requests JSON-formatted data from external sources, parses the received data to identify the type and location of the facilities, and stores this information in the database.

[0167] Step 2:

[0168] The server analyzes location data acquired using machine learning techniques. The input is location information structured in step 1, and the output is the precise location information of identified elevators and multi-purpose restrooms. Specifically, this involves the server executing algorithms using programming languages ​​such as Python and R, and performing classification and clustering by applying the data to a trained model.

[0169] Step 3:

[0170] The terminal obtains optimized location information provided by the server and provides it to the user. The input is updated location information from the server, and the output is visual map information displayed on the terminal. The terminal communicates with the server to update the map application and visualize barrier-free facilities on the map. In this process, the terminal performs specific actions to draw a particular route in response to the user's input.

[0171] Step 4:

[0172] The device uses an emotion engine to recognize the user's emotional state in real time. The input is facial images and audio data acquired through the camera and microphone, and the output is the analyzed emotional state. Specifically, the device sends images captured by the camera to a machine learning model, uses facial recognition technology to determine emotions such as joy, anger, sadness, and happiness, and then reflects the results in the overall system processing.

[0173] Step 5:

[0174] The device proposes the optimal travel route based on the user's emotional state and the specified route. The input consists of the user's emotional data and information about the origin and destination, while the output is a route proposal optimized according to the emotional state. This step involves the device using the output of the emotional engine to present the best option from multiple routes that prioritize ease of travel and safety.

[0175] Step 6:

[0176] The device monitors changes in the user's emotional state while they are on the move and communicates with a server to update information in real time. The input is new emotional state information acquired during the move, and the output is readjusted route guidance. Specifically, the device periodically evaluates the captured emotional data, retrieves the latest route information from the server as needed, and adjusts the navigation accordingly.

[0177] (Application Example 2)

[0178] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0179] The problem that this invention aims to solve is that it is difficult for users with mobility limitations to travel comfortably and safely to their destinations. In particular, since users may experience stress and anxiety during travel depending on their emotional state, route guidance provided without considering this does not sufficiently improve user convenience.

[0180] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0181] In this invention, the server includes means for collecting location information of barrier-free facilities from a database containing geographic information, means for analyzing the collected location information using artificial intelligence technology to identify the locations of lifting devices and multipurpose sanitary facilities, and means for recognizing the user's emotional state using an emotion recognition engine and optimizing the route accordingly. This makes it possible to present the optimal barrier-free route according to the user's emotions.

[0182] A "database containing geographic information" is a source of information that stores location information of barrier-free facilities and uses this information to optimize travel.

[0183] "Location information" refers to data that indicates the geographical location of specific barrier-free facilities or related equipment.

[0184] "Artificial intelligence technology" is a technology that analyzes large amounts of data and recognizes specific patterns to provide highly accurate results.

[0185] A "lifting device" refers to a mechanical device used to move users to different floors, such as an elevator or escalator.

[0186] A "multipurpose sanitary facility" is a facility that has toilets and washrooms that can be used by a variety of users.

[0187] An "emotion recognition engine" is a system that analyzes data such as a user's facial expressions and voice to identify their emotional state.

[0188] "Optimizing the route" refers to selecting the most suitable travel path according to the user's specific needs and circumstances.

[0189] The system for implementing this invention consists of a database containing geographic information, artificial intelligence technology, and an emotion recognition engine. The server first collects location information of barrier-free facilities from public institutions and related facilities and stores it in the database. This database includes location information of lifting devices, multi-purpose sanitation facilities, and other such facilities.

[0190] Next, the server uses artificial intelligence technology to analyze this location information and pinpoint the precise geographical location of a specific facility. This requires algorithms to process large amounts of geographic data quickly and reliably. The emotion recognition engine analyzes facial expressions and voice data transmitted from the user's device to recognize the user's emotional state in real time. Based on this information, the device proposes an optimized route for the user.

[0191] Specifically, the terminal uses an emotion recognition engine to identify the user's emotional state, such as stress or anxiety, based on the departure and destination points entered by the user. This information is then used to retrieve the most comfortable and safe route from the server. The server updates the information in real time accordingly and provides route guidance to the user's terminal in a visualized format. For example, if an elderly person is feeling anxious, safe and user-friendly directions can be provided based on their emotional state, making it easier for them to access public elevators or multi-purpose sanitation facilities.

[0192] As an example of a prompt, it is possible to provide information to a generative AI model using natural language processing in the form of, "Consider a route plan that guides anxious users along safe paths when they visit a new city." This enables detailed mobility assistance tailored to the individual needs of each user.

[0193] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0194] Step 1:

[0195] The server collects location information for barrier-free facilities from multiple public institutions and related facilities and stores it in a database. The input is geographical information, which is used to construct a list of barrier-free facilities. The output is the database containing the precise location information of the facilities.

[0196] Step 2:

[0197] The server uses artificial intelligence technology to analyze the collected location information and identify the locations of the lifting devices and multi-purpose sanitation facilities. The input is the location information collected in step 1, and by analyzing this, the specific location of each facility is identified. The output is accurate geographical coordinates.

[0198] Step 3:

[0199] The device uses an emotion recognition engine to analyze the user's facial expressions and voice data to identify their emotional state. The input is the user's real-time facial expressions and voice, which are then analyzed to recognize their emotional state. The output includes emotional information such as anxiety and stress.

[0200] Step 4:

[0201] The terminal requests an optimized route from the server based on the user's inputted origin and destination, taking into account their emotional state. The input consists of the user's emotional state and destination information; this information is used to request a more comfortable route. The output is optimized route information.

[0202] Step 5:

[0203] The server receives a request, optimizes the route based on the emotional state, and sends it to the terminal. The input is a route request from the terminal, and based on this, it generates different route options and selects the best one. The output is route guidance information to be provided to the terminal.

[0204] Step 6:

[0205] The terminal visually provides the user with route guidance information received from the server. The input is optimized route information, which is displayed in an easy-to-understand format. The output provides visualized information that allows the user to start their journey with confidence.

[0206] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0207] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0208] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0209] [Second Embodiment]

[0210] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0211] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0212] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0213] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0214] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0215] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0216] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0217] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0218] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0219] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0220] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0221] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0222] This invention is a system that provides users with mobility limitations with location information of barrier-free facilities based on geographic information. Specifically, a server collects geographic information and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact location of elevators and multi-purpose restrooms on a map.

[0223] The terminal retrieves updated accessibility information from the server and presents it to the user visually. Once the user enters their departure and destination points, the terminal suggests the most suitable accessibility route based on the specified path. Furthermore, the terminal updates information in real time, keeping the user informed of the latest conditions during their journey. This allows users to constantly monitor elevator availability and the availability of accessible restrooms, enabling safe and comfortable travel.

[0224] As a concrete example, consider the case of a wheelchair user using a train station in a major metropolitan area. The user can select their desired station on a terminal and check the location of barrier-free facilities. The terminal displays elevator icons on a map and provides associated route information. Furthermore, as the user progresses, the terminal issues warnings and alerts, such as notifying the user of information about temporary facility stoppages or congestion. In this way, the present invention provides specific technical means to support the smooth movement of users.

[0225] The following describes the processing flow.

[0226] Step 1:

[0227] The server collects accessibility-related information, such as elevators and multi-purpose restrooms, from APIs of public institutions and related facilities. The server stores this information in a database and updates it regularly.

[0228] Step 2:

[0229] The server uses artificial intelligence technology to analyze the information stored in the database and identify geographical information and the locations of barrier-free facilities. The identified information is stored in an optimized format to enable efficient access.

[0230] Step 3:

[0231] When a user launches an app, the device retrieves the latest accessibility information from the server. The device caches this information, making it immediately available for use.

[0232] Step 4:

[0233] The user enters their departure and destination points into the terminal's interface. Based on this information, the terminal calculates the optimal route, taking into account barrier-free facilities.

[0234] Step 5:

[0235] The terminal displays the calculated route on a map and visually indicates icons for elevators and accessible restrooms. Users can then review and select their route based on this information.

[0236] Step 6:

[0237] While in transit, the device periodically retrieves real-time information from the server and notifies the user of any changes or alerts. This allows the user to change their route as needed or continue traveling safely.

[0238] Step 7:

[0239] When a user completes a journey, the terminal collects data such as the route taken and the time spent, and sends this feedback to the server for future optimization. This allows the system to be continuously improved.

[0240] (Example 1)

[0241] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0242] Providing accurate and timely accessibility information based on geographical data remains a complex challenge for those with mobility limitations. In particular, the lack of systems that can respond immediately to real-time information changes is a significant problem. Therefore, there is a need to provide necessary information in a timely and visually easy-to-understand manner, supporting the development of safe and efficient routes.

[0243] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0244] In this invention, the server includes means for collecting location information of barrier-free facilities from an information aggregate including geographic information, means for analyzing the collected location information using computer learning technology to identify the locations of elevators and multipurpose facilities, and means for updating information on unavailability and changes in congestion in real time and adapting the information to the user's movements. As a result, users can always obtain the latest barrier-free information and enjoy an environment in which they can move around with peace of mind.

[0245] "Geographic information" refers to various types of data related to a specific location, including information such as location, coordinates, and the arrangement of facilities.

[0246] "Information aggregation" refers to a database or similar system that centrally collects data and structures it into a usable format.

[0247] "Barrier-free facilities" refer to facilities designed to be easily used by people with mobility limitations, and specifically include elevators and multi-purpose restrooms.

[0248] "Computer learning technology" refers to computer technology that analyzes data, identifies patterns, and automatically makes predictions and decisions.

[0249] "Lifting equipment" refers to mechanical equipment used to move between different floors within a building, and generally refers to elevators.

[0250] A "multipurpose facility" refers to a facility designed to be suitable for various uses, and specifically includes facilities such as wheelchair-accessible restrooms.

[0251] "User" refers to an individual or their supporter who uses this system to obtain and move information.

[0252] "Real-time updates" refers to a process that ensures information is updated instantly and always remains up-to-date.

[0253] This invention is a system that provides users with mobility limitations with location information for barrier-free facilities based on geographical data. Three entities—a server, a terminal, and a user—work together to propose the optimal travel route to the user through the processes of information collection, analysis, and provision.

[0254] The server collects information from public institutions and related facilities, gathering geographical and accessibility information. Web crawling technology and APIs can be used for this process. The collected data is stored using a relational database management system for later analysis. MySQL and PostgreSQL are suitable databases for this purpose.

[0255] Next, the server uses machine learning techniques to analyze the data. It utilizes machine learning libraries such as TensorFlow to accurately pinpoint the locations of elevators and multi-purpose facilities. Through processes such as data cleansing, feature extraction, and predictive model execution, it generates appropriate location information.

[0256] The terminal receives the latest data from the server and provides information to the user visually. To this end, it utilizes the Google Maps API to display information on a map. The locations of elevators and multi-purpose facilities are shown as icons, and route information is provided according to the user's needs. The terminal also updates new information obtained in real time according to the user's actions and provides appropriate notifications.

[0257] The user enters their departure and destination points into the terminal, and the system generates the optimal barrier-free route based on that information. This ensures that users can always travel based on the latest information. For example, by entering a prompt such as, "Tell me the barrier-free route from Tokyo Station to Shibuya Station," the terminal will provide the most suitable barrier-free information for the specified route.

[0258] This system aims to provide a technical framework that allows users to move around public spaces safely and to reduce barriers to access.

[0259] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0260] Step 1:

[0261] The server collects geographical and accessibility information from public institutions and related facilities. The input for this step is data obtained from each facility's official website or public API. For specific data processing, web crawling technology is used to extract text data and format it into a specific format. This results in output that can be stored in a database.

[0262] Step 2:

[0263] The server stores the collected information in a database. The input is the accessibility information formatted in Step 1. Here, a relational database management system is used to store the data in the appropriate tables. The output is structured data that can be used for searching and analysis.

[0264] Step 3:

[0265] The server analyzes information in the database using computer learning techniques. The input is accessibility location information stored in the database. The server trains a machine learning model and uses it to pinpoint the exact locations of elevators and multi-purpose facilities. Specific operations include data cleansing and feature extraction. The output is a dataset with location information and prediction confidence scores added.

[0266] Step 4:

[0267] The terminal retrieves parsed information from the server and prepares it for the user. The input is location data obtained in step 3. The terminal uses the Google Maps API to visually display barrier-free facilities on a map. Specifically, it uses icons and labels to indicate the location of facilities. The output is visual information on an interface that is easy for the user to view.

[0268] Step 5:

[0269] The user enters their origin and destination into the terminal and receives suggestions for the most suitable barrier-free route. The input consists of geographical selection information provided by the user. The terminal uses Dijkstra's algorithm or the A algorithm to search for the specified route and calculate the optimal route. The output is the recommended route presented to the user and its detailed information.

[0270] Step 6:

[0271] The device updates information based on new data obtained in real time. Inputs include push notifications from the server and feed information from external data sources. Specific actions in this step include retrieving data via API and updating local data. As output, the user is provided with alerts regarding the latest congestion status and facility availability.

[0272] (Application Example 1)

[0273] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0274] For users with mobility limitations, choosing safe and comfortable routes in public spaces and over long distances can be challenging. In particular, a lack of information on accessible barrier-free facilities often leads to inconvenience and anxiety during travel. Furthermore, the lack of real-time optimization of route selection results in wasted time and effort. Solutions to these problems are urgently needed.

[0275] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0276] In this invention, the server includes means for collecting location information of barrier-free facilities from information storage means including geographic information; means for analyzing the collected location information using machine learning technology to identify the locations of vertical movement devices and multi-purpose spaces; means for providing barrier-free information to users based on a specified route in an automatically controlled moving medium; and means for optimizing the travel route in cooperation with the automatically controlled moving medium. As a result, users with mobility limitations can reach their destination safely and efficiently using a barrier-free route optimized in real time.

[0277] A "means of storing information including geographic information" refers to a database system that stores spatial information such as map data and geographic coordinates in digital format and makes it accessible as needed.

[0278] A "barrier-free facility" is a building or facility designed to be safely used by people with mobility limitations, and includes facilities such as elevators and multi-purpose restrooms.

[0279] "Machine learning technology" is an information processing technology that analyzes large amounts of data, learns patterns and regularities from it, and automatically makes optimal decisions and predictions.

[0280] A "vertical movement device" is a device that enables vertical movement both inside and outside a building, and typically refers to elevators or stairlifts.

[0281] A "multipurpose space" is a space equipped with facilities that can accommodate various purposes, and a multipurpose restroom is a typical example of this.

[0282] An "automatically controlled mobile medium" refers to a vehicle or device that can move autonomously or semi-autonomously, such as an autonomous vehicle.

[0283] The "barrier-free route optimized in real time" refers to the safest and most efficient route provided to users based on the latest information at all times during movement.

[0284] This invention is a system that provides a barrier-free movement route useful when users with movement restrictions utilize an automatically controlled moving medium. This system consists of a server, a terminal, and a user.

[0285] First, the server utilizes an "information storage means including geographical information" to collect the location information of relevant barrier-free facilities. This includes digital map data and geographical coordinates. Next, it analyzes the collected information using "machine learning technology" to identify the innate location and usage status of facilities. As a result, the necessary information is provided to the automatically controlled moving medium in a constantly updated form.

[0286] The terminal visualizes this information as a "barrier-free route optimized in real time" and provides it to the user. In this process, the terminal functions as a user interface, and the information is displayed in an easy-to-see manner by an information processing device. The user can grasp in real time the distance to the destination, the current traffic conditions, etc. through the terminal.

[0287] As an example, when a wheelchair user moves within a large city in an autonomous vehicle, this system visually proposes an appropriate route based on the user's current location and destination. Also, when there is a need to update information such as the availability of elevators or the necessity to change routes, it immediately updates the information and presents it to the user. The technologies used in this system include the Google Maps API and systems that support machine learning technology, etc.

[0288] Examples of prompt sentences are as follows:

[0289] "You are a designer of an application that assists users with mobility limitations. Advise us on how to provide real-time information on accessible routes and facilities to ensure safe and comfortable travel for users of autonomous vehicles to their destinations."

[0290] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0291] Step 1:

[0292] The server collects location information for barrier-free facilities from information storage systems that include geographic information. It uses stored map data and lists of barrier-free facilities as input. Using this information, it accesses a database and extracts location information for vertical mobility devices and multi-purpose spaces. The output is structured location data for barrier-free facilities.

[0293] Step 2:

[0294] The server analyzes location information obtained using machine learning techniques. Inputs include location data obtained in Step 1 and historical usage data. A machine learning algorithm is executed to predict elevator operation status and the availability of multi-purpose spaces. The output is a real-time updated list of available barrier-free facilities.

[0295] Step 3:

[0296] The server generates optimal barrier-free route information based on the specified route. Input includes the user's origin and destination, and real-time traffic data. A route calculation algorithm is applied to this information to identify the optimal route. The output is detailed barrier-free route information provided to the user.

[0297] Step 4:

[0298] The terminal visualizes the barrier-free route information received from the server and displays it to the user. The input includes the route information identified in step 3. A map application is used to visually display the route clearly on the screen. The output is a user-friendly map display.

[0299] Step 5:

[0300] The user selects an accessible route via their device and begins their journey. Inputs include the displayed route and real-time facility usage information. During the journey, the user follows the planned route guided by the application. The output is a smooth route that ensures the user safely reaches their destination.

[0301] Step 6:

[0302] The device updates information in real time while the user is on the move and notifies them. Inputs include changes in traffic conditions and sudden changes in facility usage. It acquires new information and updates the user's route selection as needed. As a result, the user can continue to travel optimally based on the latest information. Outputs are the updated, most recent travel route and warning information.

[0303] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0304] This invention combines a system that provides users with mobility limitations with location information of barrier-free facilities based on geographical data, and an emotion engine that recognizes the user's emotions. Specifically, a server collects geographical and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact locations of elevators and accessible restrooms on a map. This helps in planning travel routes.

[0305] The terminal acquires the barrier-free information updated from the server and provides it to the user in a visually understandable form. Furthermore, using an emotion engine, it recognizes the user's emotional state in real time. When the user inputs a departure location and a destination, the terminal proposes an optimized barrier-free route according to the stress level and emotional state based on the specified route.

[0306] As a specific example, consider the case where an elderly user uses a station in a big city that they are not familiar with. The user selects the desired station and route on the terminal, and the emotion engine analyzes the user's emotional state from the user's expression and voice input. For example, when the user feels anxious, the emotion engine adjusts the route and guidance expression that can make the user feel more at ease. Also, when the emotional state changes during the movement, the terminal senses the change and communicates with the server to update the information. As a result, the user can receive detailed guidance according to individual needs and achieve a comfortable movement. This system aims to reduce stress in real time and improve the user experience, and provides a technology that promotes social harmony.

[0307] The processing flow will be described below.

[0308] Step 1:

[0309] The server collects geographical information and information on barrier-free facilities using APIs provided by public institutions and facilities. The collected information is stored in a database, and detailed mapping data including location information is prepared.

[0310] Step 2:

[0311] The server uses artificial intelligence technology to analyze the information stored in the database and identify the locations of elevators, slopes, and multi-purpose toilets in the station building and urban areas. Through this process, the interrelationship between geographical information and barrier-free facilities is accurately mapped.

[0312] Step 3:

[0313] When the app is launched, the device retrieves the latest accessibility information from the server and caches it locally. This prepares the device to display information in real time.

[0314] Step 4:

[0315] The user enters their starting point and destination into the app. Based on this information, the device calculates the optimal travel route and displays it on a map in an easy-to-understand visual format.

[0316] Step 5:

[0317] The emotion engine built into the device uses the user's camera footage and voice input to analyze the user's emotional state in real time. This process measures the user's stress and anxiety levels.

[0318] Step 6:

[0319] The device customizes the suggested routes and information it presents based on the user's emotional state. For example, if the user is feeling anxious, the device prioritizes presenting simple and intuitive routes and strives to provide clear explanations.

[0320] Step 7:

[0321] While the user is on the move, the emotion engine continuously monitors the user's state, and if it detects a change, the device will modify the guidance content as needed or send a request to the server to obtain the latest information.

[0322] Step 8:

[0323] Upon reaching the destination, the device evaluates the user's travel experience and sends the accumulated data as feedback to the server. The server analyzes this data to help improve the algorithm.

[0324] (Example 2)

[0325] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0326] It is necessary to provide safe and comfortable means of transportation for users with mobility limitations, such as people with disabilities and the elderly. However, current systems have limited access to barrier-free information and cannot suggest routes that take into account the emotional state of users, which can potentially cause stress. To solve this problem, a system is needed that updates information in real time and provides guidance tailored to the individual needs of users.

[0327] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0328] In this invention, the server includes means for acquiring location data of facilities for people with disabilities from a storage device containing geographic information, means for analyzing the acquired location data using machine learning techniques to identify the locations of elevators and multi-purpose equipment within the vehicle, and means for providing information for people with disabilities to the user based on the selected route. This makes it possible to provide a safe and comfortable travel experience by recognizing the user's emotional state and proposing optimized route suggestions in real time.

[0329] A "storage device containing geographic information" is a device that stores location data and related information about multiple facilities and places, and is used by users to obtain the desired location information.

[0330] "Facilities for people with disabilities" refers to equipment and places that have been designed or modified to be easily accessible to people with disabilities or the elderly, and includes, for example, barrier-free elevators and multi-purpose restrooms.

[0331] "Location data" refers to data that includes geographical coordinate information for a specific place or facility, and is used to identify its location on a map.

[0332] "Machine learning techniques" are technologies that automate specific tasks or decisions by having computers analyze large amounts of data and learn patterns and regularities.

[0333] "Vehicle-based lifting device" refers to a vertical movement device used to move between floors within a building, and primarily refers to an elevator.

[0334] "Multipurpose facilities" refer to facilities with a combination of functions that can accommodate various uses, and specifically include restrooms that are accessible to wheelchair users.

[0335] "User" refers to an individual who uses this system or service, and primarily includes people with disabilities and the elderly.

[0336] "Recognizing emotional states" refers to identifying and understanding the user's emotions from information such as facial expressions and voice, and is a function that is realized through technologies such as emotion engines.

[0337] "To suggest" refers to presenting a user with specific options or actions, and is an act of helping the user make the best decision.

[0338] In an embodiment for carrying out this invention, the system is configured as follows.

[0339] Server Functions

[0340] The server collects geographic information from public institutions and related facilities and stores it primarily in storage devices. This geographic information includes location data for facilities accessible to people with disabilities. To obtain this data, the server uses technology to download information via API communication. For example, the server receives data in JSON format and stores it in a database. The stored data is analyzed using machine learning techniques to identify the locations of specific facilities accessible to people with disabilities, such as elevators and accessible restrooms. Programming languages ​​such as Python and R, and their data analysis libraries, are used.

[0341] Terminal role

[0342] The terminal retrieves the latest location information from the server and provides it to the user. To this end, the terminal has the functionality to display a map application, providing information that is visually easy to understand and accessible for people with disabilities. Users can input their current location and destination using a smartphone or dedicated mobile device and receive the results. Furthermore, the terminal is equipped with an emotion engine that analyzes the user's facial expressions and voice in real time through the camera and microphone to recognize their emotional state. It uses a machine learning model to determine emotions and suggest the optimal route. It also makes adjustments to reduce the user's stress based on their emotional state, such as anxiety or reassurance.

[0343] Specific example

[0344] For example, when an elderly user visits an unfamiliar city, the user enters their starting point and destination into the device. The device reads the user's facial expressions through its built-in camera, and if the emotion engine detects that the user is anxious, it suggests a route that responds to that anxiety. Specifically, it can recommend a route that prioritizes the use of elevators over escalators.

[0345] Example of a prompt

[0346] "Please propose a reassuring guidance method to provide when users feel anxious."

[0347] This configuration allows the system to understand user needs in real time and provide a comfortable and safe travel experience.

[0348] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0349] Step 1:

[0350] The server retrieves location data for facilities accessible to people with disabilities from a storage device containing geographic information. The input is raw data obtained via API communication, and the output is structured location information stored in a database. At this stage, the server periodically requests JSON-formatted data from external sources, parses the received data to identify the type and location of the facilities, and stores this information in the database.

[0351] Step 2:

[0352] The server analyzes location data acquired using machine learning techniques. The input is location information structured in step 1, and the output is the precise location information of identified elevators and multi-purpose restrooms. Specifically, this involves the server executing algorithms using programming languages ​​such as Python and R, and performing classification and clustering by applying the data to a trained model.

[0353] Step 3:

[0354] The terminal obtains optimized location information provided by the server and provides it to the user. The input is updated location information from the server, and the output is visual map information displayed on the terminal. The terminal communicates with the server to update the map application and visualize barrier-free facilities on the map. In this process, the terminal performs specific actions to draw a particular route in response to the user's input.

[0355] Step 4:

[0356] The device uses an emotion engine to recognize the user's emotional state in real time. The input is facial images and audio data acquired through the camera and microphone, and the output is the analyzed emotional state. Specifically, the device sends images captured by the camera to a machine learning model, uses facial recognition technology to determine emotions such as joy, anger, sadness, and happiness, and then reflects the results in the overall system processing.

[0357] Step 5:

[0358] The device proposes the optimal travel route based on the user's emotional state and the specified route. The input consists of the user's emotional data and information about the origin and destination, while the output is a route proposal optimized according to the emotional state. This step involves the device using the output of the emotional engine to present the best option from multiple routes that prioritize ease of travel and safety.

[0359] Step 6:

[0360] The device monitors changes in the user's emotional state while they are on the move and communicates with a server to update information in real time. The input is new emotional state information acquired during the move, and the output is readjusted route guidance. Specifically, the device periodically evaluates the captured emotional data, retrieves the latest route information from the server as needed, and adjusts the navigation accordingly.

[0361] (Application Example 2)

[0362] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0363] The problem that this invention aims to solve is that it is difficult for users with mobility limitations to travel comfortably and safely to their destinations. In particular, since users may experience stress and anxiety during travel depending on their emotional state, route guidance provided without considering this does not sufficiently improve user convenience.

[0364] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0365] In this invention, the server includes means for collecting location information of barrier-free facilities from a database containing geographic information, means for analyzing the collected location information using artificial intelligence technology to identify the locations of lifting devices and multipurpose sanitary facilities, and means for recognizing the user's emotional state using an emotion recognition engine and optimizing the route accordingly. This makes it possible to present the optimal barrier-free route according to the user's emotions.

[0366] A "database containing geographic information" is a source of information that stores location information of barrier-free facilities and uses this information to optimize travel.

[0367] "Location information" refers to data that indicates the geographical location of specific barrier-free facilities or related equipment.

[0368] "Artificial intelligence technology" is a technology that analyzes large amounts of data and recognizes specific patterns to provide highly accurate results.

[0369] A "lifting device" refers to a mechanical device used to move users to different floors, such as an elevator or escalator.

[0370] A "multipurpose sanitary facility" is a facility that has toilets and washrooms that can be used by a variety of users.

[0371] An "emotion recognition engine" is a system that analyzes data such as a user's facial expressions and voice to identify their emotional state.

[0372] "Optimizing the route" refers to selecting the most suitable travel path according to the user's specific needs and circumstances.

[0373] The system for implementing this invention consists of a database containing geographic information, artificial intelligence technology, and an emotion recognition engine. The server first collects location information of barrier-free facilities from public institutions and related facilities and stores it in the database. This database includes location information of lifting devices, multi-purpose sanitation facilities, and other such facilities.

[0374] Next, the server uses artificial intelligence technology to analyze this location information and pinpoint the precise geographical location of a specific facility. This requires algorithms to process large amounts of geographic data quickly and reliably. The emotion recognition engine analyzes facial expressions and voice data transmitted from the user's device to recognize the user's emotional state in real time. Based on this information, the device proposes an optimized route for the user.

[0375] Specifically, the terminal uses an emotion recognition engine to identify the user's emotional state, such as stress or anxiety, based on the departure and destination points entered by the user. This information is then used to retrieve the most comfortable and safe route from the server. The server updates the information in real time accordingly and provides route guidance to the user's terminal in a visualized format. For example, if an elderly person is feeling anxious, safe and user-friendly directions can be provided based on their emotional state, making it easier for them to access public elevators or multi-purpose sanitation facilities.

[0376] As an example of a prompt, it is possible to provide information to a generative AI model using natural language processing in the form of, "Consider a route plan that guides anxious users along safe paths when they visit a new city." This enables detailed mobility assistance tailored to the individual needs of each user.

[0377] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0378] Step 1:

[0379] The server collects location information for barrier-free facilities from multiple public institutions and related facilities and stores it in a database. The input is geographical information, which is used to construct a list of barrier-free facilities. The output is the database containing the precise location information of the facilities.

[0380] Step 2:

[0381] The server uses artificial intelligence technology to analyze the collected location information and identify the locations of the lifting devices and multi-purpose sanitation facilities. The input is the location information collected in step 1, and by analyzing this, the specific location of each facility is identified. The output is accurate geographical coordinates.

[0382] Step 3:

[0383] The device uses an emotion recognition engine to analyze the user's facial expressions and voice data to identify their emotional state. The input is the user's real-time facial expressions and voice, which are then analyzed to recognize their emotional state. The output includes emotional information such as anxiety and stress.

[0384] Step 4:

[0385] The terminal requests an optimized route from the server based on the user's inputted origin and destination, taking into account their emotional state. The input consists of the user's emotional state and destination information; this information is used to request a more comfortable route. The output is optimized route information.

[0386] Step 5:

[0387] The server receives a request, optimizes the route based on the emotional state, and sends it to the terminal. The input is a route request from the terminal, and based on this, it generates different route options and selects the best one. The output is route guidance information to be provided to the terminal.

[0388] Step 6:

[0389] The terminal visually provides the user with route guidance information received from the server. The input is optimized route information, which is displayed in an easy-to-understand format. The output provides visualized information that allows the user to start their journey with confidence.

[0390] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0391] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0392] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0393] [Third Embodiment]

[0394] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0395] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0396] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0397] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0398] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0399] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0400] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0401] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0402] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0403] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0404] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0405] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0406] This invention is a system that provides users with mobility limitations with location information of barrier-free facilities based on geographic information. Specifically, a server collects geographic information and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact location of elevators and multi-purpose restrooms on a map.

[0407] The terminal retrieves updated accessibility information from the server and presents it to the user visually. Once the user enters their departure and destination points, the terminal suggests the most suitable accessibility route based on the specified path. Furthermore, the terminal updates information in real time, keeping the user informed of the latest conditions during their journey. This allows users to constantly monitor elevator availability and the availability of accessible restrooms, enabling safe and comfortable travel.

[0408] As a concrete example, consider the case of a wheelchair user using a train station in a major metropolitan area. The user can select their desired station on a terminal and check the location of barrier-free facilities. The terminal displays elevator icons on a map and provides associated route information. Furthermore, as the user progresses, the terminal issues warnings and alerts, such as notifying the user of information about temporary facility stoppages or congestion. In this way, the present invention provides specific technical means to support the smooth movement of users.

[0409] The following describes the processing flow.

[0410] Step 1:

[0411] The server collects accessibility-related information, such as elevators and multi-purpose restrooms, from APIs of public institutions and related facilities. The server stores this information in a database and updates it regularly.

[0412] Step 2:

[0413] The server uses artificial intelligence technology to analyze the information stored in the database and identify geographical information and the locations of barrier-free facilities. The identified information is stored in an optimized format to enable efficient access.

[0414] Step 3:

[0415] When a user launches an app, the device retrieves the latest accessibility information from the server. The device caches this information, making it immediately available for use.

[0416] Step 4:

[0417] The user enters their departure and destination points into the terminal's interface. Based on this information, the terminal calculates the optimal route, taking into account barrier-free facilities.

[0418] Step 5:

[0419] The terminal displays the calculated route on a map and visually indicates icons for elevators and accessible restrooms. Users can then review and select their route based on this information.

[0420] Step 6:

[0421] While in transit, the device periodically retrieves real-time information from the server and notifies the user of any changes or alerts. This allows the user to change their route as needed or continue traveling safely.

[0422] Step 7:

[0423] When a user completes a journey, the terminal collects data such as the route taken and the time spent, and sends this feedback to the server for future optimization. This allows the system to be continuously improved.

[0424] (Example 1)

[0425] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0426] Providing accurate and timely accessibility information based on geographical data remains a complex challenge for those with mobility limitations. In particular, the lack of systems that can respond immediately to real-time information changes is a significant problem. Therefore, there is a need to provide necessary information in a timely and visually easy-to-understand manner, supporting the development of safe and efficient routes.

[0427] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0428] In this invention, the server includes means for collecting location information of barrier-free facilities from an information aggregate including geographic information, means for analyzing the collected location information using computer learning technology to identify the locations of elevators and multipurpose facilities, and means for updating information on unavailability and changes in congestion in real time and adapting the information to the user's movements. As a result, users can always obtain the latest barrier-free information and enjoy an environment in which they can move around with peace of mind.

[0429] "Geographic information" refers to various types of data related to a specific location, including information such as location, coordinates, and the arrangement of facilities.

[0430] "Information aggregation" refers to a database or similar system that centrally collects data and structures it into a usable format.

[0431] "Barrier-free facilities" refer to facilities designed to be easily used by people with mobility limitations, and specifically include elevators and multi-purpose restrooms.

[0432] "Computer learning technology" refers to computer technology that analyzes data, identifies patterns, and automatically makes predictions and decisions.

[0433] "Lifting equipment" refers to mechanical equipment used to move between different floors within a building, and generally refers to elevators.

[0434] A "multipurpose facility" refers to a facility designed to be suitable for various uses, and specifically includes facilities such as wheelchair-accessible restrooms.

[0435] "User" refers to an individual or their supporter who uses this system to obtain and move information.

[0436] "Real-time updates" refers to a process that ensures information is updated instantly and always remains up-to-date.

[0437] This invention is a system that provides users with mobility limitations with location information for barrier-free facilities based on geographical data. Three entities—a server, a terminal, and a user—work together to propose the optimal travel route to the user through the processes of information collection, analysis, and provision.

[0438] The server collects information from public institutions and related facilities, gathering geographical and accessibility information. Web crawling technology and APIs can be used for this process. The collected data is stored using a relational database management system for later analysis. MySQL and PostgreSQL are suitable databases for this purpose.

[0439] Next, the server uses machine learning techniques to analyze the data. It utilizes machine learning libraries such as TensorFlow to accurately pinpoint the locations of elevators and multi-purpose facilities. Through processes such as data cleansing, feature extraction, and predictive model execution, it generates appropriate location information.

[0440] The terminal receives the latest data from the server and provides information to the user visually. To this end, it utilizes the Google Maps API to display information on a map. The locations of elevators and multi-purpose facilities are shown as icons, and route information is provided according to the user's needs. The terminal also updates new information obtained in real time according to the user's actions and provides appropriate notifications.

[0441] The user enters their departure and destination points into the terminal, and the system generates the optimal barrier-free route based on that information. This ensures that users can always travel based on the latest information. For example, by entering a prompt such as, "Tell me the barrier-free route from Tokyo Station to Shibuya Station," the terminal will provide the most suitable barrier-free information for the specified route.

[0442] This system aims to provide a technical framework that allows users to move around public spaces safely and to reduce barriers to access.

[0443] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0444] Step 1:

[0445] The server collects geographical and accessibility information from public institutions and related facilities. The input for this step is data obtained from each facility's official website or public API. For specific data processing, web crawling technology is used to extract text data and format it into a specific format. This results in output that can be stored in a database.

[0446] Step 2:

[0447] The server stores the collected information in a database. The input is the accessibility information formatted in Step 1. Here, a relational database management system is used to store the data in the appropriate tables. The output is structured data that can be used for searching and analysis.

[0448] Step 3:

[0449] The server analyzes information in the database using computer learning techniques. The input is accessibility location information stored in the database. The server trains a machine learning model and uses it to pinpoint the exact locations of elevators and multi-purpose facilities. Specific operations include data cleansing and feature extraction. The output is a dataset with location information and prediction confidence scores added.

[0450] Step 4:

[0451] The terminal retrieves parsed information from the server and prepares it for the user. The input is location data obtained in step 3. The terminal uses the Google Maps API to visually display barrier-free facilities on a map. Specifically, it uses icons and labels to indicate the location of facilities. The output is visual information on an interface that is easy for the user to view.

[0452] Step 5:

[0453] The user enters their origin and destination into the terminal and receives suggestions for the most suitable barrier-free route. The input consists of geographical selection information provided by the user. The terminal uses Dijkstra's algorithm or the A algorithm to search for the specified route and calculate the optimal route. The output is the recommended route presented to the user and its detailed information.

[0454] Step 6:

[0455] The device updates information based on new data obtained in real time. Inputs include push notifications from the server and feed information from external data sources. Specific actions in this step include retrieving data via API and updating local data. As output, the user is provided with alerts regarding the latest congestion status and facility availability.

[0456] (Application Example 1)

[0457] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0458] For users with mobility limitations, choosing safe and comfortable routes in public spaces and over long distances can be challenging. In particular, a lack of information on accessible barrier-free facilities often leads to inconvenience and anxiety during travel. Furthermore, the lack of real-time optimization of route selection results in wasted time and effort. Solutions to these problems are urgently needed.

[0459] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0460] In this invention, the server includes means for collecting location information of barrier-free facilities from information storage means including geographic information; means for analyzing the collected location information using machine learning technology to identify the locations of vertical movement devices and multi-purpose spaces; means for providing barrier-free information to users based on a specified route in an automatically controlled moving medium; and means for optimizing the travel route in cooperation with the automatically controlled moving medium. As a result, users with mobility limitations can reach their destination safely and efficiently using a barrier-free route optimized in real time.

[0461] A "means of storing information including geographic information" refers to a database system that stores spatial information such as map data and geographic coordinates in digital format and makes it accessible as needed.

[0462] A "barrier-free facility" is a building or facility designed to be safely used by people with mobility limitations, and includes facilities such as elevators and multi-purpose restrooms.

[0463] "Machine learning technology" is an information processing technology that analyzes large amounts of data, learns patterns and regularities from it, and automatically makes optimal decisions and predictions.

[0464] A "vertical movement device" is a device that enables vertical movement both inside and outside a building, and typically refers to elevators or stairlifts.

[0465] A "multipurpose space" is a space equipped with facilities that can accommodate various purposes, and a multipurpose restroom is a typical example of this.

[0466] An "automatically controlled mobile medium" refers to a vehicle or device that can move autonomously or semi-autonomously, such as an autonomous vehicle.

[0467] A "real-time optimized barrier-free route" refers to the safest and most efficient route provided to users based on the latest information at all times during their journey.

[0468] This invention provides a barrier-free travel route that is useful for users with mobility limitations when using an automated mobile medium. The system consists of a server, a terminal, and a user.

[0469] The server first uses "information storage means including geographic information" to collect location information of relevant barrier-free facilities. This includes digital map data and geographic coordinates. Next, it analyzes the collected information using "machine learning technology" to identify the inherent location and usage status of the facilities. As a result, the necessary information is provided in a constantly updated form on the automatically controlled mobile medium.

[0470] The terminal visualizes this information as a "real-time optimized barrier-free route" and provides it to the user. In this process, the terminal functions as a user interface, and the information is displayed in an easy-to-understand manner by the information processing device. Through the terminal, users can understand the distance to their destination and the current traffic conditions in real time.

[0471] For example, when a wheelchair user travels in a large city using an autonomous vehicle, this system visually suggests an appropriate route based on the user's current location and destination. Furthermore, it instantly updates and displays information regarding elevator availability and any necessary route changes. The technology used in this system includes the Google Maps API and systems supporting machine learning.

[0472] Examples of prompt statements are as follows:

[0473] "You are a designer of an application that assists users with mobility limitations. Advise us on how to provide real-time information on accessible routes and facilities to ensure safe and comfortable travel for users of autonomous vehicles to their destinations."

[0474] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0475] Step 1:

[0476] The server collects location information for barrier-free facilities from information storage systems that include geographic information. It uses stored map data and lists of barrier-free facilities as input. Using this information, it accesses a database and extracts location information for vertical mobility devices and multi-purpose spaces. The output is structured location data for barrier-free facilities.

[0477] Step 2:

[0478] The server analyzes location information obtained using machine learning techniques. Inputs include location data obtained in Step 1 and historical usage data. A machine learning algorithm is executed to predict elevator operation status and the availability of multi-purpose spaces. The output is a real-time updated list of available barrier-free facilities.

[0479] Step 3:

[0480] The server generates optimal barrier-free route information based on the specified route. Input includes the user's origin and destination, and real-time traffic data. A route calculation algorithm is applied to this information to identify the optimal route. The output is detailed barrier-free route information provided to the user.

[0481] Step 4:

[0482] The terminal visualizes the barrier-free route information received from the server and displays it to the user. The input includes the route information identified in step 3. A map application is used to visually display the route clearly on the screen. The output is a user-friendly map display.

[0483] Step 5:

[0484] The user selects an accessible route via their device and begins their journey. Inputs include the displayed route and real-time facility usage information. During the journey, the user follows the planned route guided by the application. The output is a smooth route that ensures the user safely reaches their destination.

[0485] Step 6:

[0486] The device updates information in real time while the user is on the move and notifies them. Inputs include changes in traffic conditions and sudden changes in facility usage. It acquires new information and updates the user's route selection as needed. As a result, the user can continue to travel optimally based on the latest information. Outputs are the updated, most recent travel route and warning information.

[0487] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0488] This invention combines a system that provides users with mobility limitations with location information of barrier-free facilities based on geographical data, and an emotion engine that recognizes the user's emotions. Specifically, a server collects geographical and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact locations of elevators and accessible restrooms on a map. This helps in planning travel routes.

[0489] The terminal retrieves updated accessibility information from the server and presents it to the user in a visually easy-to-understand format. Furthermore, it uses an emotion engine to recognize the user's emotional state in real time. When the user enters their starting point and destination, the terminal suggests an optimized accessibility route based on the specified route, taking into account the user's stress level and emotional state.

[0490] As a concrete example, consider the case of an elderly user using a train station in a large city they are unfamiliar with. The user selects their desired station and route on the terminal, and the emotion engine analyzes the user's emotional state from their facial expressions and voice input. For example, if the user is feeling anxious, the emotion engine adjusts the route and guidance to make it more reassuring. Furthermore, if the emotional state changes during the journey, the terminal detects the change and communicates with the server to update the information. As a result, the user receives detailed guidance tailored to their individual needs, enabling a comfortable journey. This system aims to reduce stress in real time, improve the user experience, and provide technology that promotes social harmony.

[0491] The following describes the processing flow.

[0492] Step 1:

[0493] The server collects geographical information and information on barrier-free facilities using APIs provided by public institutions and facilities. The collected information is stored in a database, and detailed mapping data, including location information, is compiled.

[0494] Step 2:

[0495] The server uses artificial intelligence technology to analyze information stored in the database and identify the locations of elevators, ramps, and accessible restrooms within train stations and urban areas. This process accurately maps the relationship between geographical information and barrier-free facilities.

[0496] Step 3:

[0497] When the app is launched, the device retrieves the latest accessibility information from the server and caches it locally. This prepares the device to display information in real time.

[0498] Step 4:

[0499] The user enters their starting point and destination into the app. Based on this information, the device calculates the optimal travel route and displays it on a map in an easy-to-understand visual format.

[0500] Step 5:

[0501] The emotion engine built into the device uses the user's camera footage and voice input to analyze the user's emotional state in real time. This process measures the user's stress and anxiety levels.

[0502] Step 6:

[0503] The device customizes the suggested routes and information it presents based on the user's emotional state. For example, if the user is feeling anxious, the device prioritizes presenting simple and intuitive routes and strives to provide clear explanations.

[0504] Step 7:

[0505] While the user is on the move, the emotion engine continuously monitors the user's state, and if it detects a change, the device will modify the guidance content as needed or send a request to the server to obtain the latest information.

[0506] Step 8:

[0507] Upon reaching the destination, the device evaluates the user's travel experience and sends the accumulated data as feedback to the server. The server analyzes this data to help improve the algorithm.

[0508] (Example 2)

[0509] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0510] It is necessary to provide safe and comfortable means of transportation for users with mobility limitations, such as people with disabilities and the elderly. However, current systems have limited access to barrier-free information and cannot suggest routes that take into account the emotional state of users, which can potentially cause stress. To solve this problem, a system is needed that updates information in real time and provides guidance tailored to the individual needs of users.

[0511] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0512] In this invention, the server includes means for acquiring location data of facilities for people with disabilities from a storage device containing geographic information, means for analyzing the acquired location data using machine learning techniques to identify the locations of elevators and multi-purpose equipment within the vehicle, and means for providing information for people with disabilities to the user based on the selected route. This makes it possible to provide a safe and comfortable travel experience by recognizing the user's emotional state and proposing optimized route suggestions in real time.

[0513] A "storage device containing geographic information" is a device that stores location data and related information about multiple facilities and places, and is used by users to obtain the desired location information.

[0514] "Facilities for people with disabilities" refers to equipment and places that have been designed or modified to be easily accessible to people with disabilities or the elderly, and includes, for example, barrier-free elevators and multi-purpose restrooms.

[0515] "Location data" refers to data that includes geographical coordinate information for a specific place or facility, and is used to identify its location on a map.

[0516] "Machine learning techniques" are technologies that automate specific tasks or decisions by having computers analyze large amounts of data and learn patterns and regularities.

[0517] "Vehicle-based lifting device" refers to a vertical movement device used to move between floors within a building, and primarily refers to an elevator.

[0518] "Multipurpose facilities" refer to facilities with a combination of functions that can accommodate various uses, and specifically include restrooms that are accessible to wheelchair users.

[0519] "User" refers to an individual who uses this system or service, and primarily includes people with disabilities and the elderly.

[0520] "Recognizing emotional states" refers to identifying and understanding the user's emotions from information such as facial expressions and voice, and is a function that is realized through technologies such as emotion engines.

[0521] "To suggest" refers to presenting a user with specific options or actions, and is an act of helping the user make the best decision.

[0522] In an embodiment for carrying out this invention, the system is configured as follows.

[0523] Server Functions

[0524] The server collects geographic information from public institutions and related facilities and stores it primarily in storage devices. This geographic information includes location data for facilities accessible to people with disabilities. To obtain this data, the server uses technology to download information via API communication. For example, the server receives data in JSON format and stores it in a database. The stored data is analyzed using machine learning techniques to identify the locations of specific facilities accessible to people with disabilities, such as elevators and accessible restrooms. Programming languages ​​such as Python and R, and their data analysis libraries, are used.

[0525] Terminal role

[0526] The terminal retrieves the latest location information from the server and provides it to the user. To this end, the terminal has the functionality to display a map application, providing information that is visually easy to understand and accessible for people with disabilities. Users can input their current location and destination using a smartphone or dedicated mobile device and receive the results. Furthermore, the terminal is equipped with an emotion engine that analyzes the user's facial expressions and voice in real time through the camera and microphone to recognize their emotional state. It uses a machine learning model to determine emotions and suggest the optimal route. It also makes adjustments to reduce the user's stress based on their emotional state, such as anxiety or reassurance.

[0527] Specific example

[0528] For example, when an elderly user visits an unfamiliar city, the user enters their starting point and destination into the device. The device reads the user's facial expressions through its built-in camera, and if the emotion engine detects that the user is anxious, it suggests a route that responds to that anxiety. Specifically, it can recommend a route that prioritizes the use of elevators over escalators.

[0529] Example of a prompt

[0530] "Please propose a reassuring guidance method to provide when users feel anxious."

[0531] This configuration allows the system to understand user needs in real time and provide a comfortable and safe travel experience.

[0532] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0533] Step 1:

[0534] The server retrieves location data for facilities accessible to people with disabilities from a storage device containing geographic information. The input is raw data obtained via API communication, and the output is structured location information stored in a database. At this stage, the server periodically requests JSON-formatted data from external sources, parses the received data to identify the type and location of the facilities, and stores this information in the database.

[0535] Step 2:

[0536] The server analyzes location data acquired using machine learning techniques. The input is location information structured in step 1, and the output is the precise location information of identified elevators and multi-purpose restrooms. Specifically, this involves the server executing algorithms using programming languages ​​such as Python and R, and performing classification and clustering by applying the data to a trained model.

[0537] Step 3:

[0538] The terminal obtains optimized location information provided by the server and provides it to the user. The input is updated location information from the server, and the output is visual map information displayed on the terminal. The terminal communicates with the server to update the map application and visualize barrier-free facilities on the map. In this process, the terminal performs specific actions to draw a particular route in response to the user's input.

[0539] Step 4:

[0540] The device uses an emotion engine to recognize the user's emotional state in real time. The input is facial images and audio data acquired through the camera and microphone, and the output is the analyzed emotional state. Specifically, the device sends images captured by the camera to a machine learning model, uses facial recognition technology to determine emotions such as joy, anger, sadness, and happiness, and then reflects the results in the overall system processing.

[0541] Step 5:

[0542] The device proposes the optimal travel route based on the user's emotional state and the specified route. The input consists of the user's emotional data and information about the origin and destination, while the output is a route proposal optimized according to the emotional state. This step involves the device using the output of the emotional engine to present the best option from multiple routes that prioritize ease of travel and safety.

[0543] Step 6:

[0544] The device monitors changes in the user's emotional state while they are on the move and communicates with a server to update information in real time. The input is new emotional state information acquired during the move, and the output is readjusted route guidance. Specifically, the device periodically evaluates the captured emotional data, retrieves the latest route information from the server as needed, and adjusts the navigation accordingly.

[0545] (Application Example 2)

[0546] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0547] The problem that this invention aims to solve is that it is difficult for users with mobility limitations to travel comfortably and safely to their destinations. In particular, since users may experience stress and anxiety during travel depending on their emotional state, route guidance provided without considering this does not sufficiently improve user convenience.

[0548] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0549] In this invention, the server includes means for collecting location information of barrier-free facilities from a database containing geographic information, means for analyzing the collected location information using artificial intelligence technology to identify the locations of lifting devices and multipurpose sanitary facilities, and means for recognizing the user's emotional state using an emotion recognition engine and optimizing the route accordingly. This makes it possible to present the optimal barrier-free route according to the user's emotions.

[0550] A "database containing geographic information" is a source of information that stores location information of barrier-free facilities and uses this information to optimize travel.

[0551] "Location information" refers to data that indicates the geographical location of specific barrier-free facilities or related equipment.

[0552] "Artificial intelligence technology" is a technology that analyzes large amounts of data and recognizes specific patterns to provide highly accurate results.

[0553] A "lifting device" refers to a mechanical device used to move users to different floors, such as an elevator or escalator.

[0554] A "multipurpose sanitary facility" is a facility that has toilets and washrooms that can be used by a variety of users.

[0555] An "emotion recognition engine" is a system that analyzes data such as a user's facial expressions and voice to identify their emotional state.

[0556] "Optimizing the route" refers to selecting the most suitable travel path according to the user's specific needs and circumstances.

[0557] The system for implementing this invention consists of a database containing geographic information, artificial intelligence technology, and an emotion recognition engine. The server first collects location information of barrier-free facilities from public institutions and related facilities and stores it in the database. This database includes location information of lifting devices, multi-purpose sanitation facilities, and other such facilities.

[0558] Next, the server uses artificial intelligence technology to analyze this location information and pinpoint the precise geographical location of a specific facility. This requires algorithms to process large amounts of geographic data quickly and reliably. The emotion recognition engine analyzes facial expressions and voice data transmitted from the user's device to recognize the user's emotional state in real time. Based on this information, the device proposes an optimized route for the user.

[0559] Specifically, the terminal uses an emotion recognition engine to identify the user's emotional state, such as stress or anxiety, based on the departure and destination points entered by the user. This information is then used to retrieve the most comfortable and safe route from the server. The server updates the information in real time accordingly and provides route guidance to the user's terminal in a visualized format. For example, if an elderly person is feeling anxious, safe and user-friendly directions can be provided based on their emotional state, making it easier for them to access public elevators or multi-purpose sanitation facilities.

[0560] As an example of a prompt, it is possible to provide information to a generative AI model using natural language processing in the form of, "Consider a route plan that guides anxious users along safe paths when they visit a new city." This enables detailed mobility assistance tailored to the individual needs of each user.

[0561] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0562] Step 1:

[0563] The server collects location information for barrier-free facilities from multiple public institutions and related facilities and stores it in a database. The input is geographical information, which is used to construct a list of barrier-free facilities. The output is the database containing the precise location information of the facilities.

[0564] Step 2:

[0565] The server uses artificial intelligence technology to analyze the collected location information and identify the locations of the lifting devices and multi-purpose sanitation facilities. The input is the location information collected in step 1, and by analyzing this, the specific location of each facility is identified. The output is accurate geographical coordinates.

[0566] Step 3:

[0567] The device uses an emotion recognition engine to analyze the user's facial expressions and voice data to identify their emotional state. The input is the user's real-time facial expressions and voice, which are then analyzed to recognize their emotional state. The output includes emotional information such as anxiety and stress.

[0568] Step 4:

[0569] The terminal requests an optimized route from the server based on the user's inputted origin and destination, taking into account their emotional state. The input consists of the user's emotional state and destination information; this information is used to request a more comfortable route. The output is optimized route information.

[0570] Step 5:

[0571] The server receives a request, optimizes the route based on the emotional state, and sends it to the terminal. The input is a route request from the terminal, and based on this, it generates different route options and selects the best one. The output is route guidance information to be provided to the terminal.

[0572] Step 6:

[0573] The terminal visually provides the user with route guidance information received from the server. The input is optimized route information, which is displayed in an easy-to-understand format. The output provides visualized information that allows the user to start their journey with confidence.

[0574] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0575] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0576] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0577] [Fourth Embodiment]

[0578] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0579] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0580] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0581] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0582] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0583] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0584] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0585] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0586] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0587] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0588] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0589] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0590] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0591] This invention is a system that provides users with mobility limitations with location information of barrier-free facilities based on geographic information. Specifically, a server collects geographic information and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact location of elevators and multi-purpose restrooms on a map.

[0592] The terminal retrieves updated accessibility information from the server and presents it to the user visually. Once the user enters their departure and destination points, the terminal suggests the most suitable accessibility route based on the specified path. Furthermore, the terminal updates information in real time, keeping the user informed of the latest conditions during their journey. This allows users to constantly monitor elevator availability and the availability of accessible restrooms, enabling safe and comfortable travel.

[0593] As a concrete example, consider the case of a wheelchair user using a train station in a major metropolitan area. The user can select their desired station on a terminal and check the location of barrier-free facilities. The terminal displays elevator icons on a map and provides associated route information. Furthermore, as the user progresses, the terminal issues warnings and alerts, such as notifying the user of information about temporary facility stoppages or congestion. In this way, the present invention provides specific technical means to support the smooth movement of users.

[0594] The following describes the processing flow.

[0595] Step 1:

[0596] The server collects accessibility-related information, such as elevators and multi-purpose restrooms, from APIs of public institutions and related facilities. The server stores this information in a database and updates it regularly.

[0597] Step 2:

[0598] The server uses artificial intelligence technology to analyze the information stored in the database and identify geographical information and the locations of barrier-free facilities. The identified information is stored in an optimized format to enable efficient access.

[0599] Step 3:

[0600] When a user launches an app, the device retrieves the latest accessibility information from the server. The device caches this information, making it immediately available for use.

[0601] Step 4:

[0602] The user enters their departure and destination points into the terminal's interface. Based on this information, the terminal calculates the optimal route, taking into account barrier-free facilities.

[0603] Step 5:

[0604] The terminal displays the calculated route on a map and visually indicates icons for elevators and accessible restrooms. Users can then review and select their route based on this information.

[0605] Step 6:

[0606] While in transit, the device periodically retrieves real-time information from the server and notifies the user of any changes or alerts. This allows the user to change their route as needed or continue traveling safely.

[0607] Step 7:

[0608] When a user completes a journey, the terminal collects data such as the route taken and the time spent, and sends this feedback to the server for future optimization. This allows the system to be continuously improved.

[0609] (Example 1)

[0610] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0611] Providing accurate and timely accessibility information based on geographical data remains a complex challenge for those with mobility limitations. In particular, the lack of systems that can respond immediately to real-time information changes is a significant problem. Therefore, there is a need to provide necessary information in a timely and visually easy-to-understand manner, supporting the development of safe and efficient routes.

[0612] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0613] In this invention, the server includes means for collecting location information of barrier-free facilities from an information aggregate including geographic information, means for analyzing the collected location information using computer learning technology to identify the locations of elevators and multipurpose facilities, and means for updating information on unavailability and changes in congestion in real time and adapting the information to the user's movements. As a result, users can always obtain the latest barrier-free information and enjoy an environment in which they can move around with peace of mind.

[0614] "Geographic information" refers to various types of data related to a specific location, including information such as location, coordinates, and the arrangement of facilities.

[0615] "Information aggregation" refers to a database or similar system that centrally collects data and structures it into a usable format.

[0616] "Barrier-free facilities" refer to facilities designed to be easily used by people with mobility limitations, and specifically include elevators and multi-purpose restrooms.

[0617] "Computer learning technology" refers to computer technology that analyzes data, identifies patterns, and automatically makes predictions and decisions.

[0618] "Lifting equipment" refers to mechanical equipment used to move between different floors within a building, and generally refers to elevators.

[0619] A "multipurpose facility" refers to a facility designed to be suitable for various uses, and specifically includes facilities such as wheelchair-accessible restrooms.

[0620] "User" refers to an individual or their supporter who uses this system to obtain and move information.

[0621] "Real-time updates" refers to a process that ensures information is updated instantly and always remains up-to-date.

[0622] This invention is a system that provides users with mobility limitations with location information for barrier-free facilities based on geographical data. Three entities—a server, a terminal, and a user—work together to propose the optimal travel route to the user through the processes of information collection, analysis, and provision.

[0623] The server collects information from public institutions and related facilities, gathering geographical and accessibility information. Web crawling technology and APIs can be used for this process. The collected data is stored using a relational database management system for later analysis. MySQL and PostgreSQL are suitable databases for this purpose.

[0624] Next, the server uses machine learning techniques to analyze the data. It utilizes machine learning libraries such as TensorFlow to accurately pinpoint the locations of elevators and multi-purpose facilities. Through processes such as data cleansing, feature extraction, and predictive model execution, it generates appropriate location information.

[0625] The terminal receives the latest data from the server and provides information to the user visually. To this end, it utilizes the Google Maps API to display information on a map. The locations of elevators and multi-purpose facilities are shown as icons, and route information is provided according to the user's needs. The terminal also updates new information obtained in real time according to the user's actions and provides appropriate notifications.

[0626] The user enters their departure and destination points into the terminal, and the system generates the optimal barrier-free route based on that information. This ensures that users can always travel based on the latest information. For example, by entering a prompt such as, "Tell me the barrier-free route from Tokyo Station to Shibuya Station," the terminal will provide the most suitable barrier-free information for the specified route.

[0627] This system aims to provide a technical framework that allows users to move around public spaces safely and to reduce barriers to access.

[0628] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0629] Step 1:

[0630] The server collects geographical and accessibility information from public institutions and related facilities. The input for this step is data obtained from each facility's official website or public API. For specific data processing, web crawling technology is used to extract text data and format it into a specific format. This results in output that can be stored in a database.

[0631] Step 2:

[0632] The server stores the collected information in a database. The input is the accessibility information formatted in Step 1. Here, a relational database management system is used to store the data in the appropriate tables. The output is structured data that can be used for searching and analysis.

[0633] Step 3:

[0634] The server analyzes information in the database using computer learning techniques. The input is accessibility location information stored in the database. The server trains a machine learning model and uses it to pinpoint the exact locations of elevators and multi-purpose facilities. Specific operations include data cleansing and feature extraction. The output is a dataset with location information and prediction confidence scores added.

[0635] Step 4:

[0636] The terminal retrieves parsed information from the server and prepares it for the user. The input is location data obtained in step 3. The terminal uses the Google Maps API to visually display barrier-free facilities on a map. Specifically, it uses icons and labels to indicate the location of facilities. The output is visual information on an interface that is easy for the user to view.

[0637] Step 5:

[0638] The user enters their origin and destination into the terminal and receives suggestions for the most suitable barrier-free route. The input consists of geographical selection information provided by the user. The terminal uses Dijkstra's algorithm or the A algorithm to search for the specified route and calculate the optimal route. The output is the recommended route presented to the user and its detailed information.

[0639] Step 6:

[0640] The device updates information based on new data obtained in real time. Inputs include push notifications from the server and feed information from external data sources. Specific actions in this step include retrieving data via API and updating local data. As output, the user is provided with alerts regarding the latest congestion status and facility availability.

[0641] (Application Example 1)

[0642] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0643] For users with mobility limitations, choosing safe and comfortable routes in public spaces and over long distances can be challenging. In particular, a lack of information on accessible barrier-free facilities often leads to inconvenience and anxiety during travel. Furthermore, the lack of real-time optimization of route selection results in wasted time and effort. Solutions to these problems are urgently needed.

[0644] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0645] In this invention, the server includes means for collecting location information of barrier-free facilities from information storage means including geographic information; means for analyzing the collected location information using machine learning technology to identify the locations of vertical movement devices and multi-purpose spaces; means for providing barrier-free information to users based on a specified route in an automatically controlled moving medium; and means for optimizing the travel route in cooperation with the automatically controlled moving medium. As a result, users with mobility limitations can reach their destination safely and efficiently using a barrier-free route optimized in real time.

[0646] A "means of storing information including geographic information" refers to a database system that stores spatial information such as map data and geographic coordinates in digital format and makes it accessible as needed.

[0647] A "barrier-free facility" is a building or facility designed to be safely used by people with mobility limitations, and includes facilities such as elevators and multi-purpose restrooms.

[0648] "Machine learning technology" is an information processing technology that analyzes large amounts of data, learns patterns and regularities from it, and automatically makes optimal decisions and predictions.

[0649] A "vertical movement device" is a device that enables vertical movement both inside and outside a building, and typically refers to elevators or stairlifts.

[0650] A "multipurpose space" is a space equipped with facilities that can accommodate various purposes, and a multipurpose restroom is a typical example of this.

[0651] An "automatically controlled mobile medium" refers to a vehicle or device that can move autonomously or semi-autonomously, such as an autonomous vehicle.

[0652] A "real-time optimized barrier-free route" refers to the safest and most efficient route provided to users based on the latest information at all times during their journey.

[0653] This invention provides a barrier-free travel route that is useful for users with mobility limitations when using an automated mobile medium. The system consists of a server, a terminal, and a user.

[0654] The server first uses "information storage means including geographic information" to collect location information of relevant barrier-free facilities. This includes digital map data and geographic coordinates. Next, it analyzes the collected information using "machine learning technology" to identify the inherent location and usage status of the facilities. As a result, the necessary information is provided in a constantly updated form on the automatically controlled mobile medium.

[0655] The terminal visualizes this information as a "real-time optimized barrier-free route" and provides it to the user. In this process, the terminal functions as a user interface, and the information is displayed in an easy-to-understand manner by the information processing device. Through the terminal, users can understand the distance to their destination and the current traffic conditions in real time.

[0656] For example, when a wheelchair user travels in a large city using an autonomous vehicle, this system visually suggests an appropriate route based on the user's current location and destination. Furthermore, it instantly updates and displays information regarding elevator availability and any necessary route changes. The technology used in this system includes the Google Maps API and systems supporting machine learning.

[0657] Examples of prompt statements are as follows:

[0658] "You are a designer of an application that assists users with mobility limitations. Advise us on how to provide real-time information on accessible routes and facilities to ensure safe and comfortable travel for users of autonomous vehicles to their destinations."

[0659] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0660] Step 1:

[0661] The server collects location information for barrier-free facilities from information storage systems that include geographic information. It uses stored map data and lists of barrier-free facilities as input. Using this information, it accesses a database and extracts location information for vertical mobility devices and multi-purpose spaces. The output is structured location data for barrier-free facilities.

[0662] Step 2:

[0663] The server analyzes location information obtained using machine learning techniques. Inputs include location data obtained in Step 1 and historical usage data. A machine learning algorithm is executed to predict elevator operation status and the availability of multi-purpose spaces. The output is a real-time updated list of available barrier-free facilities.

[0664] Step 3:

[0665] The server generates optimal barrier-free route information based on the specified route. Input includes the user's origin and destination, and real-time traffic data. A route calculation algorithm is applied to this information to identify the optimal route. The output is detailed barrier-free route information provided to the user.

[0666] Step 4:

[0667] The terminal visualizes the barrier-free route information received from the server and displays it to the user. The input includes the route information identified in step 3. A map application is used to visually display the route clearly on the screen. The output is a user-friendly map display.

[0668] Step 5:

[0669] The user selects an accessible route via their device and begins their journey. Inputs include the displayed route and real-time facility usage information. During the journey, the user follows the planned route guided by the application. The output is a smooth route that ensures the user safely reaches their destination.

[0670] Step 6:

[0671] The device updates information in real time while the user is on the move and notifies them. Inputs include changes in traffic conditions and sudden changes in facility usage. It acquires new information and updates the user's route selection as needed. As a result, the user can continue to travel optimally based on the latest information. Outputs are the updated, most recent travel route and warning information.

[0672] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0673] This invention combines a system that provides users with mobility limitations with location information of barrier-free facilities based on geographical data, and an emotion engine that recognizes the user's emotions. Specifically, a server collects geographical and barrier-free information from public institutions and related facilities and stores it in a database. The server uses artificial intelligence technology to analyze this data and pinpoint the exact locations of elevators and accessible restrooms on a map. This helps in planning travel routes.

[0674] The terminal retrieves updated accessibility information from the server and presents it to the user in a visually easy-to-understand format. Furthermore, it uses an emotion engine to recognize the user's emotional state in real time. When the user enters their starting point and destination, the terminal suggests an optimized accessibility route based on the specified route, taking into account the user's stress level and emotional state.

[0675] As a concrete example, consider the case of an elderly user using a train station in a large city they are unfamiliar with. The user selects their desired station and route on the terminal, and the emotion engine analyzes the user's emotional state from their facial expressions and voice input. For example, if the user is feeling anxious, the emotion engine adjusts the route and guidance to make it more reassuring. Furthermore, if the emotional state changes during the journey, the terminal detects the change and communicates with the server to update the information. As a result, the user receives detailed guidance tailored to their individual needs, enabling a comfortable journey. This system aims to reduce stress in real time, improve the user experience, and provide technology that promotes social harmony.

[0676] The following describes the processing flow.

[0677] Step 1:

[0678] The server collects geographical information and information on barrier-free facilities using APIs provided by public institutions and facilities. The collected information is stored in a database, and detailed mapping data, including location information, is compiled.

[0679] Step 2:

[0680] The server uses artificial intelligence technology to analyze information stored in the database and identify the locations of elevators, ramps, and accessible restrooms within train stations and urban areas. This process accurately maps the relationship between geographical information and barrier-free facilities.

[0681] Step 3:

[0682] When the app is launched, the device retrieves the latest accessibility information from the server and caches it locally. This prepares the device to display information in real time.

[0683] Step 4:

[0684] The user enters their starting point and destination into the app. Based on this information, the device calculates the optimal travel route and displays it on a map in an easy-to-understand visual format.

[0685] Step 5:

[0686] The emotion engine built into the device uses the user's camera footage and voice input to analyze the user's emotional state in real time. This process measures the user's stress and anxiety levels.

[0687] Step 6:

[0688] The device customizes the suggested routes and information it presents based on the user's emotional state. For example, if the user is feeling anxious, the device prioritizes presenting simple and intuitive routes and strives to provide clear explanations.

[0689] Step 7:

[0690] While the user is on the move, the emotion engine continuously monitors the user's state, and if it detects a change, the device will modify the guidance content as needed or send a request to the server to obtain the latest information.

[0691] Step 8:

[0692] Upon reaching the destination, the device evaluates the user's travel experience and sends the accumulated data as feedback to the server. The server analyzes this data to help improve the algorithm.

[0693] (Example 2)

[0694] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0695] It is necessary to provide safe and comfortable means of transportation for users with mobility limitations, such as people with disabilities and the elderly. However, current systems have limited access to barrier-free information and cannot suggest routes that take into account the emotional state of users, which can potentially cause stress. To solve this problem, a system is needed that updates information in real time and provides guidance tailored to the individual needs of users.

[0696] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0697] In this invention, the server includes means for acquiring location data of facilities for people with disabilities from a storage device containing geographic information, means for analyzing the acquired location data using machine learning techniques to identify the locations of elevators and multi-purpose equipment within the vehicle, and means for providing information for people with disabilities to the user based on the selected route. This makes it possible to provide a safe and comfortable travel experience by recognizing the user's emotional state and proposing optimized route suggestions in real time.

[0698] A "storage device containing geographic information" is a device that stores location data and related information about multiple facilities and places, and is used by users to obtain the desired location information.

[0699] "Facilities for people with disabilities" refers to equipment and places that have been designed or modified to be easily accessible to people with disabilities or the elderly, and includes, for example, barrier-free elevators and multi-purpose restrooms.

[0700] "Location data" refers to data that includes geographical coordinate information for a specific place or facility, and is used to identify its location on a map.

[0701] "Machine learning techniques" are technologies that automate specific tasks or decisions by having computers analyze large amounts of data and learn patterns and regularities.

[0702] "Vehicle-based lifting device" refers to a vertical movement device used to move between floors within a building, and primarily refers to an elevator.

[0703] "Multipurpose facilities" refer to facilities with a combination of functions that can accommodate various uses, and specifically include restrooms that are accessible to wheelchair users.

[0704] "User" refers to an individual who uses this system or service, and primarily includes people with disabilities and the elderly.

[0705] "Recognizing emotional states" refers to identifying and understanding the user's emotions from information such as facial expressions and voice, and is a function that is realized through technologies such as emotion engines.

[0706] "To suggest" refers to presenting a user with specific options or actions, and is an act of helping the user make the best decision.

[0707] In an embodiment for carrying out this invention, the system is configured as follows.

[0708] Server Functions

[0709] The server collects geographic information from public institutions and related facilities and stores it primarily in storage devices. This geographic information includes location data for facilities accessible to people with disabilities. To obtain this data, the server uses technology to download information via API communication. For example, the server receives data in JSON format and stores it in a database. The stored data is analyzed using machine learning techniques to identify the locations of specific facilities accessible to people with disabilities, such as elevators and accessible restrooms. Programming languages ​​such as Python and R, and their data analysis libraries, are used.

[0710] Terminal role

[0711] The terminal retrieves the latest location information from the server and provides it to the user. To this end, the terminal has the functionality to display a map application, providing information that is visually easy to understand and accessible for people with disabilities. Users can input their current location and destination using a smartphone or dedicated mobile device and receive the results. Furthermore, the terminal is equipped with an emotion engine that analyzes the user's facial expressions and voice in real time through the camera and microphone to recognize their emotional state. It uses a machine learning model to determine emotions and suggest the optimal route. It also makes adjustments to reduce the user's stress based on their emotional state, such as anxiety or reassurance.

[0712] Specific example

[0713] For example, when an elderly user visits an unfamiliar city, the user enters their starting point and destination into the device. The device reads the user's facial expressions through its built-in camera, and if the emotion engine detects that the user is anxious, it suggests a route that responds to that anxiety. Specifically, it can recommend a route that prioritizes the use of elevators over escalators.

[0714] Example of a prompt

[0715] "Please propose a reassuring guidance method to provide when users feel anxious."

[0716] This configuration allows the system to understand user needs in real time and provide a comfortable and safe travel experience.

[0717] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0718] Step 1:

[0719] The server retrieves location data for facilities accessible to people with disabilities from a storage device containing geographic information. The input is raw data obtained via API communication, and the output is structured location information stored in a database. At this stage, the server periodically requests JSON-formatted data from external sources, parses the received data to identify the type and location of the facilities, and stores this information in the database.

[0720] Step 2:

[0721] The server analyzes location data acquired using machine learning techniques. The input is location information structured in step 1, and the output is the precise location information of identified elevators and multi-purpose restrooms. Specifically, this involves the server executing algorithms using programming languages ​​such as Python and R, and performing classification and clustering by applying the data to a trained model.

[0722] Step 3:

[0723] The terminal obtains optimized location information provided by the server and provides it to the user. The input is updated location information from the server, and the output is visual map information displayed on the terminal. The terminal communicates with the server to update the map application and visualize barrier-free facilities on the map. In this process, the terminal performs specific actions to draw a particular route in response to the user's input.

[0724] Step 4:

[0725] The device uses an emotion engine to recognize the user's emotional state in real time. The input is facial images and audio data acquired through the camera and microphone, and the output is the analyzed emotional state. Specifically, the device sends images captured by the camera to a machine learning model, uses facial recognition technology to determine emotions such as joy, anger, sadness, and happiness, and then reflects the results in the overall system processing.

[0726] Step 5:

[0727] The device proposes the optimal travel route based on the user's emotional state and the specified route. The input consists of the user's emotional data and information about the origin and destination, while the output is a route proposal optimized according to the emotional state. This step involves the device using the output of the emotional engine to present the best option from multiple routes that prioritize ease of travel and safety.

[0728] Step 6:

[0729] The device monitors changes in the user's emotional state while they are on the move and communicates with a server to update information in real time. The input is new emotional state information acquired during the move, and the output is readjusted route guidance. Specifically, the device periodically evaluates the captured emotional data, retrieves the latest route information from the server as needed, and adjusts the navigation accordingly.

[0730] (Application Example 2)

[0731] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0732] The problem that this invention aims to solve is that it is difficult for users with mobility limitations to travel comfortably and safely to their destinations. In particular, since users may experience stress and anxiety during travel depending on their emotional state, route guidance provided without considering this does not sufficiently improve user convenience.

[0733] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0734] In this invention, the server includes means for collecting location information of barrier-free facilities from a database containing geographic information, means for analyzing the collected location information using artificial intelligence technology to identify the locations of lifting devices and multipurpose sanitary facilities, and means for recognizing the user's emotional state using an emotion recognition engine and optimizing the route accordingly. This makes it possible to present the optimal barrier-free route according to the user's emotions.

[0735] A "database containing geographic information" is a source of information that stores location information of barrier-free facilities and uses this information to optimize travel.

[0736] "Location information" refers to data that indicates the geographical location of specific barrier-free facilities or related equipment.

[0737] "Artificial intelligence technology" is a technology that analyzes large amounts of data and recognizes specific patterns to provide highly accurate results.

[0738] A "lifting device" refers to a mechanical device used to move users to different floors, such as an elevator or escalator.

[0739] A "multipurpose sanitary facility" is a facility that has toilets and washrooms that can be used by a variety of users.

[0740] An "emotion recognition engine" is a system that analyzes data such as a user's facial expressions and voice to identify their emotional state.

[0741] "Optimizing the route" refers to selecting the most suitable travel path according to the user's specific needs and circumstances.

[0742] The system for implementing this invention consists of a database containing geographic information, artificial intelligence technology, and an emotion recognition engine. The server first collects location information of barrier-free facilities from public institutions and related facilities and stores it in the database. This database includes location information of lifting devices, multi-purpose sanitation facilities, and other such facilities.

[0743] Next, the server uses artificial intelligence technology to analyze this location information and pinpoint the precise geographical location of a specific facility. This requires algorithms to process large amounts of geographic data quickly and reliably. The emotion recognition engine analyzes facial expressions and voice data transmitted from the user's device to recognize the user's emotional state in real time. Based on this information, the device proposes an optimized route for the user.

[0744] Specifically, the terminal uses an emotion recognition engine to identify the user's emotional state, such as stress or anxiety, based on the departure and destination points entered by the user. This information is then used to retrieve the most comfortable and safe route from the server. The server updates the information in real time accordingly and provides route guidance to the user's terminal in a visualized format. For example, if an elderly person is feeling anxious, safe and user-friendly directions can be provided based on their emotional state, making it easier for them to access public elevators or multi-purpose sanitation facilities.

[0745] As an example of a prompt, it is possible to provide information to a generative AI model using natural language processing in the form of, "Consider a route plan that guides anxious users along safe paths when they visit a new city." This enables detailed mobility assistance tailored to the individual needs of each user.

[0746] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0747] Step 1:

[0748] The server collects location information for barrier-free facilities from multiple public institutions and related facilities and stores it in a database. The input is geographical information, which is used to construct a list of barrier-free facilities. The output is the database containing the precise location information of the facilities.

[0749] Step 2:

[0750] The server uses artificial intelligence technology to analyze the collected location information and identify the locations of the lifting devices and multi-purpose sanitation facilities. The input is the location information collected in step 1, and by analyzing this, the specific location of each facility is identified. The output is accurate geographical coordinates.

[0751] Step 3:

[0752] The device uses an emotion recognition engine to analyze the user's facial expressions and voice data to identify their emotional state. The input is the user's real-time facial expressions and voice, which are then analyzed to recognize their emotional state. The output includes emotional information such as anxiety and stress.

[0753] Step 4:

[0754] The terminal requests an optimized route from the server based on the user's inputted origin and destination, taking into account their emotional state. The input consists of the user's emotional state and destination information; this information is used to request a more comfortable route. The output is optimized route information.

[0755] Step 5:

[0756] The server receives a request, optimizes the route based on the emotional state, and sends it to the terminal. The input is a route request from the terminal, and based on this, it generates different route options and selects the best one. The output is route guidance information to be provided to the terminal.

[0757] Step 6:

[0758] The terminal visually provides the user with route guidance information received from the server. The input is optimized route information, which is displayed in an easy-to-understand format. The output provides visualized information that allows the user to start their journey with confidence.

[0759] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0760] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0761] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0762] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0763] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0764] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0765] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0766] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0767] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0768] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0769] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0770] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0771] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0772] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0773] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0774] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0775] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0776] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0777] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0778] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0779] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0780] The following is further disclosed regarding the embodiments described above.

[0781] (Claim 1)

[0782] A means for collecting location information of barrier-free facilities from a database containing geographic information,

[0783] A means of using artificial intelligence technology to analyze collected location information and identify the locations of elevators and multipurpose restrooms,

[0784] A means of providing accessibility information to users based on a designated route,

[0785] A means of updating information in real time and supporting users' movement,

[0786] A system that includes this.

[0787] (Claim 2)

[0788] The system according to claim 1, which visualizes barrier-free information and displays it in a visually easy-to-understand manner on the user's terminal.

[0789] (Claim 3)

[0790] The system according to claim 1, which presents multiple route options and selects the optimal route based on the comfort of travel.

[0791] "Example 1"

[0792] (Claim 1)

[0793] A means of collecting location information of barrier-free facilities from an aggregate of information including geographic information,

[0794] A means for analyzing collected location information using computer learning technology to identify the location of the elevator and multipurpose facility,

[0795] A means of providing accessibility information to users based on a designated route,

[0796] A means of updating information on unavailability and changes in congestion in real time, and adapting the information to the movements of users.

[0797] A system that includes this.

[0798] (Claim 2)

[0799] The system according to claim 1, which visualizes barrier-free information and displays it on the user's device in a way that is easy to understand intuitively.

[0800] (Claim 3)

[0801] The system according to claim 1, which presents multiple route options and selects the optimal route based on the comfort of travel.

[0802] "Application Example 1"

[0803] (Claim 1)

[0804] A means for collecting location information of barrier-free facilities from information storage means that include geographic information,

[0805] A means for analyzing collected location information using machine learning technology to identify the location of vertical movement devices and multi-purpose spaces,

[0806] In an automated mobile medium, means for providing barrier-free information to users based on a specified route,

[0807] A means of updating information in real time and supporting users' movement,

[0808] A means for optimizing the movement path in conjunction with an automatically controlled moving medium,

[0809] A system that includes this.

[0810] (Claim 2)

[0811] The system according to claim 1, which visualizes barrier-free information and displays it in a visually easy-to-understand manner on the user's information processing device.

[0812] (Claim 3)

[0813] The system according to claim 1, which presents multiple route options and selects the optimal route based on comfort and efficiency of travel.

[0814] "Example 2 of combining an emotion engine"

[0815] (Claim 1)

[0816] A means for acquiring location data of facilities for people with disabilities from a storage device that includes geographic information,

[0817] A means for analyzing acquired position data using machine learning techniques to identify the location of the elevator and multi-purpose equipment inside the vehicle,

[0818] A means of providing users with disability-friendly information based on the selected route,

[0819] A means of supporting the user's movement by updating information as time passes,

[0820] A means for recognizing the user's emotional state and providing an optimized route suggestion based on that state,

[0821] A system that includes this.

[0822] (Claim 2)

[0823] The system according to claim 1, which visualizes information for people with disabilities and displays it on the user's terminal in a way that is easy to understand visually.

[0824] (Claim 3)

[0825] The system according to claim 1, which detects the emotional state of the user and selects the optimal route based on the user's level of safety during travel.

[0826] "Application example 2 when combining with an emotional engine"

[0827] (Claim 1)

[0828] A means for collecting location information of barrier-free facilities from a database containing geographic information,

[0829] A means of analyzing collected location information using artificial intelligence technology to identify the location of the lifting device and the multi-purpose sanitation facility,

[0830] A means of providing accessibility information to users based on a designated route,

[0831] A means for recognizing the user's emotional state using an emotion recognition engine and optimizing the route accordingly,

[0832] A means of updating information in real time and supporting users' movement,

[0833] A system that includes this.

[0834] (Claim 2)

[0835] The system according to claim 1, which visualizes barrier-free information and displays it in a visually easy-to-understand manner on the user's information processing device.

[0836] (Claim 3)

[0837] The system according to claim 1, which presents multiple route options and selects the optimal route based on the comfort of travel and the user's emotional state. [Explanation of Symbols]

[0838] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for collecting location information of barrier-free facilities from a database containing geographic information, A means of analyzing collected location information using artificial intelligence technology to identify the locations of elevators and multipurpose restrooms, A means of providing accessibility information to users based on a designated route, A means of updating information in real time and supporting users' movement, A system that includes this.

2. The system according to claim 1, which visualizes barrier-free information and displays it in a visually easy-to-understand manner on the user's terminal.

3. The system according to claim 1, which presents multiple route options and selects the optimal route based on the comfort of travel.

Images