System

Abstract

To provide a safe and easy-to-use transportation system for solving problems accompanying the going out and movement of an aged person.SOLUTION: A system comprising: a user terminal operated by a resident for outing support; a server that receives and manages a request transmitted from the user terminal; a robot that autonomously moves based on an instruction transmitted from the server and connects an electric mobility scooter; the electric mobility scooter for moving the resident in a state of being connected to the robot; means for detecting an obstacle by a sensor mounted on the robot and safely driving the robot; and means for remotely controlling an operation of the robot by a control signal transmitted from the server.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] There is a problem that elderly people have limited opportunities to go out due to concerns about their legs and back, making it difficult for them to participate in society. This can lead to a deterioration in health and an increase in feelings of loneliness, which can reduce the quality of their daily lives. Furthermore, it can be difficult for them to move around within their apartment complex, and they often have to rely on other residents and managers, which can cause stress and inconvenience. To solve these problems associated with elderly people going out and traveling, there is a growing need for a safe and easy-to-use transportation system. [Means for solving the problem]

[0005] The above-mentioned problem is solved by a system that includes a user device operated by residents to assist them in going out; a server that receives and manages requests sent from the user device; a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car; an electric senior car that transports residents while connected to the robot; a means for detecting obstacles using sensors installed on the robot to ensure safe operation; and a means for remotely controlling the robot's operation using control signals sent from the server. Specifically, users can input transportation requests via a smartphone app, which the server receives and manages and sends instructions to the appropriate robot. The robot moves autonomously and connects to the electric senior car to transport the user safely. The robot is equipped with the latest sensor technology and AI, enabling obstacle detection, facial recognition, and safe driving, making it safe for seniors to use. This will increase opportunities for seniors to go out, promoting health maintenance and social participation.

[0006] "Residents" refers to the residents of the apartment complex who use the service, particularly the elderly.

[0007] "User terminal" refers to devices such as smartphones and tablets operated by residents.

[0008] "Server" refers to the central control device that receives and manages requests sent from user terminals and sends instructions to robots.

[0009] The "robot" refers to an autonomous mobile security robot that moves autonomously and is connected to an electric senior car to transport residents to and from work.

[0010] An "electric senior car" refers to an electric transportation device that is connected to a robot and allows residents to sit and move around.

[0011] "Sensor" refers to an electronic component installed on a robot that detects obstacles.

[0012] "AI technology" refers to the artificial intelligence technology used to enable robots to communicate smoothly with residents and operate appropriately.

[0013] A "transportation request" refers to a request for assistance with going out that is input by a resident via a user terminal and transmitted to the server.

[0014] A "control signal" is instruction data sent from the server to the robot, and refers to a signal used to remotely control the robot's operations.

[0015] "Obstacle detection" refers to the robot's ability to use sensors to detect obstacles in its surroundings and avoid collisions. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that supports residents in moving from their homes safely and efficiently, and is realized by linking user terminals, servers, robots, and electric senior cars.

[0038] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[0039] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereinafter referred to as the robot). The server manages the user's location information and request content, and monitors the robot's location and status in real time.

[0040] The robot receives instructions from the server and heads to the designated entrance lobby, where it connects the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. AI technology is also used to enable smooth communication with residents. For example, the robot can notify the user of its arrival time in advance and provide directions along the way. The robot connects the senior car and heads to the user's entrance lobby.

[0041] The user gets into the senior car in the entrance lobby and travels with the robot to their room in their home. During this time, the robot uses sensors and AI technology to drive safely, and can also accept remote control via control signals sent from the server as needed. The user's device can check the robot's location in real time, allowing the user to travel with peace of mind.

[0042] As a concrete example, we will show a scenario in which an elderly resident, Mr. A, wants to return to his room at home after going out.

[0043] 1. User (A):

[0044] Open the smartphone app and enter your pickup request.

[0045] 2. Device (app):

[0046] Send A's request to the server.

[0047] 3. Server:

[0048] Upon receiving the request, confirm Mr. A's location and his room number at home.

[0049] It instructs the nearest robot to move to the entrance lobby and connect the senior cart.

[0050] 4. Robots:

[0051] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[0052] 5. Robot and Person A:

[0053] We welcome Mr. A in the entrance lobby and guide him to the senior car.

[0054] After checking for safety, I head to my room.

[0055] 6. User Device:

[0056] The robot's location is displayed in real time, providing Mr. A with a sense of security.

[0057] 7. Arrival:

[0058] We arrive in front of Mr. A's house and park the mobility scooter.

[0059] The robot notifies the server that the transfer is complete.

[0060] This system provides an environment where elderly people can move around the apartment building safely and efficiently, and is expected to promote social participation and maintain health. This invention is a practical means to improve the quality of life for elderly people and also brings benefits to apartment building managers.

[0061] The processing flow will be explained below.

[0062] Step 1:

[0063] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[0064] Step 2:

[0065] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[0066] Step 3:

[0067] The terminal transmits the pickup request input by the user to the server, including the user's location information and destination information.

[0068] Step 4:

[0069] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[0070] Step 5:

[0071] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[0072] Step 6:

[0073] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[0074] Step 7:

[0075] When the robot arrives at the entrance lobby, it will notify the user of its arrival via voice guidance.

[0076] Step 8:

[0077] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[0078] Step 9:

[0079] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[0080] Step 10:

[0081] The terminal transmits the user's "depart" instruction to the server.

[0082] Step 11:

[0083] The server sends a "go" command to the robot.

[0084] Step 12:

[0085] The robot receives the "depart" command and begins autonomously moving towards the room in the home. It uses its built-in sensors to detect obstacles and drive safely.

[0086] Step 13:

[0087] The terminal obtains the robot's location information in real time and provides it to the user.

[0088] Step 14:

[0089] When the robot arrives at its destination, it will notify the user by voice.

[0090] Step 15:

[0091] The user gets out of the mobility scooter and enters a room in their home.

[0092] Step 16:

[0093] The robot notifies the server that the transfer is complete.

[0094] Step 17:

[0095] The server marks the robot's task as completed and checks for the robot's next task.

[0096] Step 18:

[0097] The robot moves to an appropriate position to wait for the next instruction if there is no next task.

[0098] This series of processes will result in a system that will enable safe and secure transportation for the elderly.

[0099] Example 1

[0100] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0101] In modern society, there is a demand for support systems that enable elderly people and people with disabilities to travel efficiently and safely. However, current systems lack the technical capabilities to provide real-time location information, detect obstacles for safe driving, and provide autonomous mobility support. This makes it difficult to provide an environment where elderly people and people with disabilities can go out with peace of mind.

[0102] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0103] In this invention, the server includes a data analysis means for analyzing requests from the information processing device and issuing appropriate instructions, a mechanism means for the mechanical device to move in response to instructions and connect to the mobile device at a specified location, and a mechanical device that moves autonomously and connects to the mobile device based on instructions sent from the data processing device, thereby enabling elderly people and people with disabilities to move efficiently and safely.

[0104] "User" refers to a person who uses the system to receive mobility assistance.

[0105] The term "information processing device" refers to a terminal device operated by a user, which inputs and transmits requests.

[0106] "Data processing device" refers to a central system that receives and analyzes requests sent from an information processing device and issues instructions.

[0107] The term "mechanical device" refers to a device such as a robot that operates autonomously based on instructions from a data processing device and that supports the movement of a user by connecting to a mobile device.

[0108] "Mobility device" refers to a device such as an electric senior car or wheelchair that is connected to a mechanical device to transport a user.

[0109] "Sensor" refers to a device installed on a machine to detect obstacles and sense the surrounding environment.

[0110] "Data analysis means" refers to a method or technology for analyzing a request sent from an information processing device and creating appropriate instructions.

[0111] The term "mechanical means" refers to a mechanism by which a mechanical device connects a moving device based on an instruction.

[0112] "Artificial intelligence technology" refers to machine learning and natural language processing technologies used by machines to smoothly communicate with users.

[0113] This invention is a system that supports the elderly and physically disabled to move around safely and efficiently. The system consists of a user terminal, a data processing device (server), a mechanical device (robot), and a mobility device (electric senior car). The specific hardware and software configurations, as well as data processing and calculation methods, are explained below.

[0114] Hardware and software used

[0115] User terminal: Includes information processing devices such as smartphones and tablets, on which dedicated applications are installed.

[0116] Data Processing Unit (Server): The central system that analyzes requests and issues appropriate instructions to the robot. It includes a database and a communication interface.

[0117] Mechanical device (robot): A robot capable of autonomous movement, equipped with high-precision sensors and artificial intelligence (AI) technology.

[0118] Mobility device (electric senior car): A means of transporting the user in cooperation with the robot. It operates by being connected to the robot.

[0119] Program processing overview

[0120] User device operation

[0121] Users input and send their transportation requests using a dedicated application, which includes functions such as inputting requests, obtaining current locations, and displaying real-time location information.

[0122] Server Processing

[0123] The server analyzes requests received from the user's device and confirms the user's current location and destination. Based on the analysis results, it issues instructions to the nearest robot, including moving to the entrance lobby and connecting to an electric senior car. The server also monitors the robot's location and status in real time and sends control signals as necessary.

[0124] Robot movement

[0125] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves and connects to the electric senior car. Upon arrival, the robot greets the user and guides them to the senior car. During the journey, it makes full use of AI technology to check for safety and communicates smoothly with the user as needed.

[0126] Specific example explanation

[0127] When elderly resident A goes out and wants to return to his room, the following steps are taken:

[0128] 1. User: Opens the smartphone app and enters a pickup request.

[0129] Example prompt: How do I enter a destination using my smartphone?

[0130] 2. Terminal: The user terminal sends a request to the server.

[0131] Sample prompt: Describe the process a user goes through to submit a trip request in your app.

[0132] 3. Server: Receives the request, analyzes it, and sends instructions to the nearest robot.

[0133] Sample prompt: Please explain in detail how the server parses the request.

[0134] 4. Robot: Following instructions, it heads to the entrance lobby and connects the electric senior car.

[0135] Sample prompt: How can the robot navigate to the designated location and connect the mobility scooter?

[0136] 5. Robot: Welcomes the user in the entrance lobby and guides them to the mobility scooter.

[0137] Sample prompt: Please explain in detail the steps the robot takes to pick up the user.

[0138] This system provides an environment where elderly people and people with disabilities can move around safely and efficiently.

[0139] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0140] Step 1:

[0141] Enter your transportation request

[0142] User:

[0143] The resident opens the smartphone app, taps the "Request a Pickup" button on the app's home screen, and selects their room as the destination.

[0144] Input: Destination selection, user location information

[0145] Output: Pickup request information (user ID, current location, destination, timestamp)

[0146] Step 2:

[0147] Submit a Pickup Request

[0148] Device:

[0149] The user device sends A's request information to the server. When the user taps the send button, the request information is sent to the server via the network.

[0150] Input: Pickup request information

[0151] Output: HTTP request to the server

[0152] Step 3:

[0153] Receiving and parsing the request

[0154] server:

[0155] The server receives and analyzes the request sent from the user's device. The server stores the received request information (user ID, room number, current location) in a database and assigns the nearest robot.

[0156] Input: HTTP request (transport request information)

[0157] Output: Instructions to the robot (destination, current location, user information)

[0158] Step 4:

[0159] Robot instructions

[0160] server:

[0161] The server instructs the nearest robot to move to the entrance lobby and connect to the electric senior car. The server sends an HTTP request to the robot, instructing it to move and connect to the senior car. After the robot receives the command, it begins to act.

[0162] Input: Instructions for the robot

[0163] Output: Robot starts moving

[0164] Step 5:

[0165] Robot movement and connection

[0166] robot:

[0167] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves, and the mechanism for connecting the electric senior car to it is activated.

[0168] Input: Instructions from the server

[0169] Output: Arrival at the entrance lobby and docking of the mobility scooter

[0170] Step 6:

[0171] Meet and greet users

[0172] robot:

[0173] The robot greets Mr. A in the entrance lobby and guides him to the senior car. It calls out to Mr. A by voice and guides him to the boarding position of the senior car. The robot uses AI-based natural language processing to converse with him.

[0174] Input: User pickup instructions

[0175] Output: Directions to the user's mobility scooter

[0176] Step 7:

[0177] Providing real-time information

[0178] Device:

[0179] The user's device displays the robot's real-time location information. The app displays a map on the screen, and the current locations of the robot and the user are updated in real time, giving the user a sense of security.

[0180] Input: Robot location

[0181] Output: Real-time map display

[0182] Step 8:

[0183] Completing the move

[0184] robot:

[0185] The robot and the senior car deliver Mr. A to his room at home. After arriving, the robot notifies the server that the transfer is complete. The server receives this notification and records the completion of the transfer in the database.

[0186] Input: Arrival at destination

[0187] Output: Notification of completion of transfer to server

[0188] (Application example 1)

[0189] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0190] Existing technologies for supporting elderly and disabled residents to travel safely and efficiently from their homes to their destinations face challenges in terms of safety and convenience. Furthermore, there is a lack of ways for residents to visually check real-time information while traveling, making it difficult for them to travel safely. Furthermore, technology is needed to enable smooth communication between residents and robots.

[0191] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0192] In this invention, the server includes a user terminal operated by the resident to provide assistance with going out, a means for receiving and managing requests sent from the user terminal, a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, the electric senior car for transporting the resident while connected to the robot, a means for detecting obstacles using sensors mounted on the robot and safely driving it, a means for remotely controlling the robot's operation using control signals sent from the server, and a means for visualizing the robot's position information and movement route in real time using smart glasses or a head-mounted display. This enables residents to travel safely and efficiently from their homes to their destinations with peace of mind.

[0193] A "user terminal" is an electronic device operated by residents to assist them in going out, and includes smartphones, tablets, and the like.

[0194] A "server" is a central management device that receives and manages requests sent from user terminals.

[0195] A "robot" is a mechanical device that moves autonomously based on instructions sent from a server and connects to an electric senior car.

[0196] An "electric senior car" is an electric vehicle that is connected to a robot and used to transport residents.

[0197] A "sensor" is a device installed on a robot that detects obstacles.

[0198] A "control signal" is an operation command signal sent from the server to the robot.

[0199] "Remote control means" refers to a method of remotely controlling the robot's movements using control signals sent from a server.

[0200] "Smart glasses" are wearable devices that can visually display the robot's position information and movement route.

[0201] A "head-mounted display" is a device worn by a user on the head that displays information, and is used to visualize the position information and movement path of a robot.

[0202] "Real-time information" refers to data or information that provides an immediate picture of the current situation.

[0203] "AI technology" refers to artificial intelligence technology that is installed in robots and enables smooth communication with residents.

[0204] "Visualization means" refers to a method for visualizing the robot's position information and movement path using smart glasses, a head-mounted display, or the like.

[0205] This invention is a support system for elderly people and people with disabilities to move from their homes to their destinations safely and efficiently. A specific embodiment of the system is described below.

[0206] System configuration

[0207] The system includes a user terminal, a server, a robot, an electric senior car, smart glasses, and a head-mounted display.

[0208] 1. User Device

[0209] A user terminal is an electronic device, such as a smartphone or tablet, that a resident uses to input a transportation request.

[0210] The application installed on the user terminal has a function for sending requests and a function for displaying real-time location information.

[0211] 2. Server

[0212] The server receives and analyzes the request sent from the user terminal.

[0213] The server manages the user's location information and request details, and sends instructions to the most appropriate robot.

[0214] The server is built using cloud services such as AWS, and communication is carried out in real time using WebSocket.

[0215] 3. Robot

[0216] The autonomous mobile robot moves automatically and connects to the electric senior car based on instructions from the server.

[0217] The robot is equipped with high-precision sensors (e.g., LIDAR, cameras) to detect and avoid obstacles.

[0218] Using AI technology (e.g. TensorFlow), we communicate smoothly with residents and provide guidance along the way.

[0219] 4. Electric senior car

[0220] The electric senior car is an electric vehicle connected to a robot to transport residents.

[0221] Senior cars provide safe and comfortable transportation.

[0222] 5. Smart Glasses and Head-Mounted Displays

[0223] Smart glasses or head-mounted displays (e.g., Google Glass, Microsoft HoloLens) are used to visualize the robot's position and path.

[0224] This allows residents to check the robot's status in real time and move around with peace of mind.

[0225] Example

[0226] For example, when a resident moves from their room to the entrance of an apartment building, they follow the steps below.

[0227] 1. Request transmission from user terminal

[0228] The user starts the application on their device and inputs a pickup request, which includes destination information and current location.

[0229] 2. Request Processing on the Server

[0230] The server receives the request, analyzes it, and sends instructions to the most suitable robot.

[0231] 3. Robot movement and resident guidance

[0232] The robot receives instructions, starts moving autonomously, connects to the electric senior cart, and detects and avoids obstacles as necessary.

[0233] The robot will notify the user's device or smart glasses of its arrival time and provide guidance along the way.

[0234] 4. Displaying real-time information

[0235] The robot's position and movement path are visualized through smart glasses or a head-mounted display.

[0236] Prompt example

[0237] Possible input prompts for a generative AI model include:

[0238] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[0239] This system allows residents to travel from their homes to their destinations efficiently and safely with peace of mind. This completes the embodiment of the present invention.

[0240] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0241] Step 1:

[0242] The application is started on the user terminal and a transportation request is input.

[0243] The input includes current location and destination information.

[0244] The user terminal transmits the input data to the server.

[0245] Input: Current location and destination information from the user

[0246] Output: Sending request data to the server

[0247] Step 2:

[0248] The server receives the request sent from the user terminal.

[0249] The server analyzes the request data and obtains the user's current location and destination information.

[0250] Input: Request data sent from the user terminal

[0251] Output: Parsed current location and destination information

[0252] Step 3:

[0253] The server checks the current location of the robots to select the most suitable robot.

[0254] A movement instruction is generated for the optimal robot and sent to the robot.

[0255] Input: Current location and destination information, and robot location data

[0256] Output: Movement instructions to the robot

[0257] Step 4:

[0258] The robot receives movement instructions from the server and begins moving autonomously.

[0259] The robot connects to an electric senior car and moves towards its destination.

[0260] Input: Move instruction from server

[0261] Output: Robot starts moving

[0262] Step 5:

[0263] Sensors installed on the robot detect obstacles and select a safe route for driving.

[0264] It processes sensor data in real time and performs data calculations to avoid obstacles.

[0265] Input: Sensor data from the surrounding environment

[0266] Output: Path information for obstacle avoidance

[0267] Step 6:

[0268] The robot sends its location information and estimated arrival time to the server.

[0269] The server provides this information to user devices, smart glasses, and head-mounted displays in real time.

[0270] Input: Robot location and estimated arrival time

[0271] Output: Real-time notifications to user devices and smart glasses

[0272] Step 7:

[0273] The user visually checks the robot's location and movement route using smart glasses or a head-mounted display.

[0274] This allows users to travel with peace of mind.

[0275] Input: Location information and movement route data from the server

[0276] Output: Real-time display as visual information

[0277] Step 8:

[0278] When the robot arrives at its destination, it sends an arrival notification to the server.

[0279] The server notifies the user terminal of the arrival information, and the series of movements is completed.

[0280] Input: Arrival notification from the robot

[0281] Output: Arrival information notification to user terminal

[0282] Prompt example

[0283] Possible input prompts for a generative AI model include:

[0284] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions, and is realized by linking the user terminal, server, robot, electric senior car, and emotion engine.

[0287] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[0288] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereafter referred to as the robot) based on the user's location and destination information. The server monitors the robot's location and status in real time, and sends control signals as necessary to remotely control the robot's operation.

[0289] The robot receives instructions from the server and heads to the designated entrance lobby to connect the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. It is also capable of smooth communication with residents using AI technology. It is also equipped with an emotion engine that can analyze the user's emotional state and respond appropriately. The emotion engine recognizes emotions based on the user's facial expressions, voice, and behavioral data, and the robot responds appropriately.

[0290] The specific operation of the system will be described below.

[0291] Consider a scenario where Resident A goes out and wants to return to his room at home.

[0292] 1. User (A):

[0293] Open the smartphone app and enter your pickup request.

[0294] 2. Device (app):

[0295] A request for transportation from Mr. A is sent to the server.

[0296] 3. Server:

[0297] Receive the request and confirm Mr. A's location and destination.

[0298] It instructs the nearest robot to move and perform a series of actions.

[0299] 4. Robots:

[0300] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[0301] The system uses an emotion engine to analyze Mr. A's emotional state and greets him appropriately upon arrival.

[0302] 5. User (A):

[0303] Check the senior car connected to the robot in the entrance lobby and get in.

[0304] Travel with peace of mind thanks to the robot's voice guidance.

[0305] 6. Robots:

[0306] It uses sensors to avoid obstacles and safely drive towards your room in the home.

[0307] If Mr. A seems anxious while driving, the emotion engine will detect this and reassure him with appropriate words.

[0308] 7. Terminal:

[0309] The robot's location is displayed in real time and provided to Mr. A.

[0310] 8. Robots:

[0311] When they arrive at their destination, they inform Mr. A of their arrival and greet him in a friendly manner.

[0312] 9. User (A):

[0313] I get off the senior car and enter my room at home.

[0314] 10. Robots:

[0315] The server is notified that the transfer is complete, and the emotional state of Mr. A recognized by the emotion engine is also sent.

[0316] If there is no next task, move to an appropriate waiting position.

[0317] This series of processes will realize a system that allows users to move around the apartment building safely and efficiently while taking their emotions into consideration. The emotion engine's response is expected to provide an elderly-friendly environment and improve the quality of life.

[0318] The processing flow will be explained below.

[0319] Step 1:

[0320] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[0321] Step 2:

[0322] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[0323] Step 3:

[0324] The device transmits the location information along with the pickup request entered by the user to the server, including the user's destination information.

[0325] Step 4:

[0326] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[0327] Step 5:

[0328] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[0329] Step 6:

[0330] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[0331] Step 7:

[0332] When the robot arrives at the entrance lobby, it notifies the user of its arrival with a voice prompt. It also uses a built-in emotion engine to analyze the user's emotions and provides voice prompts and greetings according to the user's emotional state.

[0333] Step 8:

[0334] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[0335] Step 9:

[0336] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[0337] Step 10:

[0338] The terminal transmits the user's "depart" instruction to the server.

[0339] Step 11:

[0340] The server sends a "go" command to the robot.

[0341] Step 12:

[0342] The robot receives the "depart" command and begins autonomously moving toward the room in the home. It uses built-in sensors to detect obstacles and drive safely. It also uses an emotion engine to monitor the user's emotions.

[0343] Step 13:

[0344] The robot analyzes the user's emotions while moving, and if it detects anxiety or tension, it will provide voice guidance and dialogue to provide comfort.

[0345] Step 14:

[0346] The terminal displays the robot's location information in real time and provides it to the user.

[0347] Step 15:

[0348] When the robot arrives at its destination, it notifies the user and speaks in a friendly manner. Based on the analysis results of its emotion engine, it checks whether the user is calm.

[0349] Step 16:

[0350] The user gets out of the mobility scooter and enters a room in their home.

[0351] Step 17:

[0352] The robot notifies the server that the transfer is complete and also sends the user's emotional state as recognized by the emotion engine.

[0353] Step 18:

[0354] The server marks the robot's task as completed and checks to see if there are any other tasks waiting.

[0355] Step 19:

[0356] If there is no next task, the robot moves to an appropriate waiting position and waits for the next instruction.

[0357] This series of processes will realize a system that utilizes an emotion engine to allow elderly people to move safely around an apartment building while also taking into consideration their emotional state.

[0358] Example 2

[0359] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0360] Conventional outing support systems simply support movement without considering the user's emotional state. As a result, elderly and emotionally unstable users often feel anxious while out and about. Another issue is that communication with the user is not smooth, making it difficult for the user to feel familiar with the system.

[0361] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a user terminal operated by the resident to provide outing assistance, means for receiving and managing requests sent from the user terminal, means including a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, an electric senior car for transporting the resident while connected to the robot, means for detecting obstacles using sensors mounted on the robot and driving safely, means for remotely controlling the operation of the robot using control signals sent from the server, means for analyzing the emotional state of the resident using an emotion engine, and means for providing appropriate greetings or calls based on the user's emotions. This enables safe and friendly outing assistance that takes the user's emotional state into consideration.

[0362] A "user terminal" is an electronic device operated by a user, and includes a smartphone, a tablet, and the like.

[0363] A "server" is a computer system that receives and manages requests sent from user terminals.

[0364] A "request" is request information sent from a user terminal to a server, and includes instructions for transportation or movement.

[0365] The "robot" is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car.

[0366] An "electric senior car" is an electric mobility device that is connected to a robot to safely transport residents.

[0367] "Sensors" are devices installed on robots to detect obstacles, and include LIDAR and cameras.

[0368] A "control signal" is an instruction sent from the server to the robot to remotely control the robot's operations.

[0369] An "emotion engine" is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response.

[0370] "Appropriate greetings and calls" are communication actions that the robot takes toward the user based on the analysis results of the emotion engine.

[0371] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions. The main components of the system are a user terminal, a server, a robot, an electric senior car, and an emotion engine, all of which work in conjunction with each other.

[0372] User terminal

[0373] A user terminal is an electronic device operated by a user, and includes smartphones, tablets, etc. A dedicated application is installed on the device, and the user uses this application to input a pickup request. For example, simply inputting a request such as "I want to go to my room at home" initiates a series of operations in the system.

[0374] server

[0375] The server is a computer system that receives and manages requests sent from the user's terminal. The server analyzes the request data and sends appropriate instructions to the robot based on the user's current location and destination. During this process, the server constantly monitors the robot's location and status in real time, and sends control signals each time to remotely control the robot's operation.

[0376] robot

[0377] The robot is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car. The robot is equipped with high-precision sensors and can move safely while detecting obstacles using LIDAR and cameras. The robot also has an emotion engine that recognizes emotions based on the user's facial expressions, voice, and behavioral data. For example, if the user appears anxious, the robot can reassure them with appropriate words such as, "Don't worry. We'll soon arrive at our destination."

[0378] Electric senior car

[0379] An electric senior car is an electric mobility device that connects to a robot to safely transport residents. Because it is controlled by the robot, the user does not need to drive it themselves. It allows residents to travel safely and smoothly to specific destinations within the apartment complex.

[0380] Emotion Engine

[0381] The emotion engine is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response. The emotion engine uses data obtained from a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing their emotional state in real time. This allows the robot to respond appropriately to the user, providing a sense of security, especially for elderly and emotionally unstable users.

[0382] Add specific examples to the description

[0383] For example, if resident A wants to return to their room after being out, the following scenario can be considered: First, resident A opens the smartphone app and enters a request for transportation. The app sends the request to the server, which receives and analyzes the request, and then sends instructions to the nearest robot. Based on instructions from the server, the robot heads to the entrance lobby and connects its electric senior car. The robot then uses its emotion engine to analyze resident A's emotional state and responds with an appropriate greeting, such as, "Hello, resident A. How are you feeling today?" resident A gets into the senior car and follows the robot's guidance, feeling reassured as they travel. Finally, when the robot arrives at its destination and resident A gets out of the senior car, the robot notifies the server that transportation is complete and waits for the next task.

[0384] Prompt Sentence Examples

[0385] Below are some example prompts for a generative AI model:

[0386] "Please explain how an autonomous mobile robot equipped with an emotion engine can analyze the emotional state of its user."

[0387] "Please explain in detail the procedure for welcoming a robot using the emotion engine."

[0388] This not only allows users to use the system with peace of mind, but also provides an environment that is particularly friendly to elderly people.The present invention aims to improve the quality of life of users.

[0389] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0390] Step 1:

[0391] The user opens a dedicated app installed on their smartphone or tablet and enters a transportation request.

[0392] Specific operation: The user launches the app, enters their destination (e.g., their room number at home), and presses the pick-up button.

[0393] Input: Trip request information entered by the user (destination, current location).

[0394] Output: Trip request data generated within the app.

[0395] Step 2:

[0396] The terminal sends the user's transportation request to the server.

[0397] Specific behavior: The app sends a request containing the user's location information (GPS data) and destination information to the server as an HTTP POST request.

[0398] Input: The generated trip request data.

[0399] Output: The request data sent to the server.

[0400] Step 3:

[0401] The server receives the request and verifies the user's location and destination.

[0402] Specific operation: The server analyzes the request data and identifies the user's current location (for example, the entrance to an apartment building).

[0403] Input: Request data sent from the device.

[0404] Output: Parsed user location and destination information.

[0405] Step 4:

[0406] The server instructs the nearest robot to move and perform a series of actions.

[0407] Specific operation: The server checks the robot's current location, calculates the optimal route, and then sends instructions such as "head to the entrance, connect the senior cart, and welcome the user."

[0408] Input: User's current location, destination, and robot's current location data.

[0409] Output: Movement and motion instruction data sent to the robot.

[0410] Step 5:

[0411] The robot receives instructions from the server, heads to the entrance lobby, and connects the electric senior car.

[0412] Specific operation: The robot uses its own navigation system to autonomously navigate to the entrance lobby and physically connect with the senior car.

[0413] Input: Movement and action instruction data sent from the server.

[0414] Output: Arrival at the entrance lobby and the connection status of the mobility scooter.

[0415] Step 6:

[0416] The robot uses an emotion engine to analyze the user's emotional state and greets them appropriately upon arrival.

[0417] Specific operation: The robot uses a camera and microphone to analyze the user's facial expressions and tone of voice, and then greets them with a greeting such as, "Hello, Mr. A. How are you feeling today?"

[0418] Input: User's facial expression data, voice data.

[0419] Output: Analysis results and a voice message based on them.

[0420] Step 7:

[0421] The user checks the senior car connected to the robot in the entrance lobby and gets in.

[0422] Specific operation: The user follows the robot's instructions to get into the senior car.

[0423] Input: User confirmation and ride operation.

[0424] Output: Riding in a mobility scooter.

[0425] Step 8:

[0426] The robot's voice guidance provides users with a sense of security as they move around.

[0427] What it does: The robot provides voice guidance such as, "Don't worry, we're on our way to your next destination."

[0428] Input: Robot voice guidance data.

[0429] Output: A voice message to the user.

[0430] Step 9:

[0431] The robot uses sensors to avoid obstacles and safely navigates towards your room.

[0432] How it works: The robot uses a LIDAR sensor to scan the surrounding environment and automatically selects a safe route to move.

[0433] Input: Sensor data, environmental data.

[0434] Output: Obstacle-avoiding route information and safe movement status.

[0435] Step 10:

[0436] If the user appears anxious while the robot is driving, the emotion engine will detect this and provide appropriate reassurance.

[0437] What it does: The emotion engine detects the user's anxiety and provides reassuring words such as, "You're almost there."

[0438] Input: User's facial expression data, voice data.

[0439] Output: Analysis results and a voice message based on them.

[0440] Step 11:

[0441] The robot arrives at the destination and notifies the user of its arrival while speaking in a friendly manner.

[0442] Specific operation: The robot will provide guidance such as, "Mr. A, you have arrived at your destination. Thank you for your hard work."

[0443] Input: Destination arrival data.

[0444] Output: A voice message to the user.

[0445] Step 12:

[0446] The user gets out of the mobility scooter and enters a room in their home.

[0447] Specific actions: The user gets out of the mobility scooter, unlocks the door to their home, and enters.

[0448] Input: User exit and entry operations.

[0449] Output: The user's room entry status.

[0450] Step 13:

[0451] The robot notifies the server that the transfer is complete and also transmits the user's emotional state as recognized by the emotion engine.

[0452] Specific operation: The robot uploads the data of the completed transfer to the server and also sends the emotion data.

[0453] Input: Pick-up and drop-off completion data, emotional state data.

[0454] Output: Data sent to the server.

[0455] Step 14:

[0456] If the robot has no next task, it moves to an appropriate waiting position.

[0457] Specific behavior: The robot moves to the waiting station and waits for further instructions.

[0458] Input: Next task availability data.

[0459] Output: Waiting in standby position.

[0460] (Application example 2)

[0461] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0462] There is a need for systems that allow elderly people and people who require physical assistance to shop safely and with peace of mind in physical stores. It is also important to provide appropriate support according to their emotional state. Existing systems have had problems with insufficient emotion recognition and are unable to adequately respond to the anxiety and confusion of elderly people.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and managing requests sent from a user terminal, means for controlling an autonomous mobile device that moves autonomously based on instructions sent from the server and connects to an electric vehicle, and means for recognizing the emotional state of residents using an emotion engine installed in the autonomous mobile device and taking appropriate action. This enables elderly people and people who require physical assistance to move safely within a store and shop with peace of mind. Furthermore, the emotion engine provides appropriate support according to the user's emotional state, thereby improving their sense of security and satisfaction.

[0464] "Residents" are users of the system, primarily elderly people and those who require physical assistance.

[0465] "User terminal" refers to electronic devices such as smartphones and tablets that are operated by residents.

[0466] A "request" refers to information sent from a user terminal to a server to request support or movement.

[0467] The term "server" refers to a computer system that manages requests sent from user terminals and sends instructions to autonomous mobile devices.

[0468] "Autonomous mobile equipment" refers to robots and similar equipment that move autonomously and connect to electric vehicles based on instructions from a server to support residents.

[0469] "Electric vehicle" refers to an electric senior car or other electrically powered vehicle that is connected to an autonomous mobility device and used to transport residents.

[0470] "Sensor" refers to a device installed on an autonomous mobile device to detect obstacles in the surrounding area.

[0471] "Emotion engine" refers to artificial intelligence technology that recognizes residents' emotional state and responds appropriately.

[0472] "Artificial intelligence technology" refers to technology used by autonomous mobile devices to facilitate communication with residents.

[0473] This invention is a system that helps elderly people and people who require physical assistance to shop safely in physical stores. The system is composed of components including a server, user terminals, autonomous mobile devices, electric vehicles, sensors, an emotion engine, and artificial intelligence technology.

[0474] First, the user device is a smartphone or tablet. Residents submit support requests through a dedicated application. This request is sent from the device to a server. The server receives the request and analyzes the resident's location and the type of support required. Based on the analysis results, it sends instructions to the nearest autonomous mobile device.

[0475] The autonomous mobile device receives instructions from the server and moves to a designated location within the store. This device connects to an electric vehicle (such as an electric cart or senior cart) to transport the resident. The autonomous mobile device is equipped with sensors that detect and avoid obstacles in the surrounding area. It is also equipped with an emotion engine that analyzes the resident's facial expressions, voice, and behavior to recognize their emotional state and respond appropriately.

[0476] The server monitors the location of the autonomous mobile device in real time and remotely controls it as needed, using a high-precision control system.

[0477] As a specific example, consider the following scenario.

[0478] Scenario: Elderly person B goes shopping at a supermarket

[0479] 1. User (Mr. B):

[0480] Open the smartphone app, select "Shopping Support," and submit a support request.

[0481] 2. Device (app):

[0482] Send Mr. B's shopping support request to the server.

[0483] 3. Server:

[0484] Receive the request and confirm Mr. B's location and the support he needs.

[0485] Send instructions to the nearest autonomous mobile device.

[0486] 4. Autonomous mobile devices:

[0487] Following the server's instructions, I headed towards the entrance of the supermarket.

[0488] The emotional engine is used to analyze Mr. B's emotional state and provide an appropriate greeting or greeting upon arrival.

[0489] 5. Person B:

[0490] The user contacts an autonomous mobile device near the entrance and requests assistance with shopping.

[0491] Follow the voice guidance of the autonomous mobile device and start shopping with peace of mind.

[0492] 6. Autonomous mobile devices:

[0493] As they move around the store together, they guide B to products based on his shopping list.

[0494] Move safely while detecting obstacles.

[0495] If Person B feels anxious, the emotion engine will detect this and provide appropriate words to reassure them.

[0496] Hardware and Software Used

[0497] User device: Smartphone (iOS / Android)

[0498] Server: Cloud server (AWS or Google Cloud) + Flask (Python web server framework)

[0499] Autonomous mobile device: Autonomous mobile robot (equipped with LiDAR sensor, camera, and microphone)

[0500] Emotion Engine: Emotion Recognition Model with TensorFlow / Keras

[0501] An example prompt is:

[0502] "Imagine that Person B goes to the supermarket and asks an autonomous mobile device to help him with his shopping. Person B is feeling a little anxious. How would the autonomous mobile device respond to reassure Person B?"

[0503] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0504] Step 1:

[0505] User (Mr. B)

[0506] Open the smartphone app, select "Shopping Support," and submit a support request.

[0507] Input: B's location information, request details

[0508] Output: Support request data

[0509] Step 2:

[0510] Device (app)

[0511] Send Mr. B's shopping support request to the server.

[0512] Input: Support request data

[0513] Output: Request data sent to the server

[0514] Step 3:

[0515] server

[0516] The system receives the request, checks B's location and the support required, analyzes the request data, and sends instructions to the nearest autonomous mobile device.

[0517] Input: Request data sent to the server

[0518] Data processing / calculation: Analyzes the request and selects the most suitable autonomous mobile device

[0519] Output: Instruction data sent to the autonomous mobile device

[0520] Step 4:

[0521] Autonomous mobile equipment

[0522] It receives instructions from the server and moves to a designated location within the store. During its movement, it uses sensors to detect surrounding obstacles and proceed safely.

[0523] Input: Instruction data sent to the autonomous mobile device

[0524] Data processing / calculation: Calculating movement routes based on location information, detecting obstacles using sensors

[0525] Output: Action instructions to move safely

[0526] Step 5:

[0527] Autonomous mobile equipment

[0528] Upon arrival, the emotion engine is used to analyze Mr. B's emotional state and an appropriate greeting or greeting is given.

[0529] Input: Person B's facial expression and voice data

[0530] Data processing / calculation: Emotion recognition using TensorFlow / Keras, and decision on how to respond based on the analysis results

[0531] Output: An appropriate response to Person B

[0532] Step 6:

[0533] User (Mr. B)

[0534] Users contact an autonomous mobile device near the entrance and request shopping assistance. They then follow the audio guidance provided by the autonomous mobile device and begin shopping with peace of mind.

[0535] Input: Autonomous mobile device information

[0536] Output: Person B's actions

[0537] Step 7:

[0538] Autonomous mobile equipment

[0539] As the robot moves around the store together with B, it guides him to products based on his shopping list. Along the way, it again uses sensors to detect and avoid obstacles. If B feels confused or anxious, the emotion engine detects this and uses appropriate words to reassure him.

[0540] Input: B's shopping list, facial expressions, voice data, sensor information

[0541] Data processing / calculation: Navigation to product locations based on shopping lists, emotion recognition, obstacle avoidance

[0542] Output: Product guidance, obstacle avoidance behavior, appropriate response

[0543] Step 8:

[0544] Autonomous mobile equipment

[0545] Once the customer has finished shopping, they will be guided to the cash register and supported until payment is complete.

[0546] Input: B's actions, cash register location information

[0547] Data processing / calculation: Secure route calculation, payment completion confirmation

[0548] Output: Instructions to cashier, necessary responses

[0549] Step 9:

[0550] server

[0551] It notifies the robot that the shopping support is complete and records the emotional state of the robot according to the emotion engine. If there is no next task, the robot moves to an appropriate waiting position.

[0552] Input: Completion notification from autonomous mobile device, emotional state data

[0553] Data processing / calculation: Confirmation of task completion, movement plan to waiting position

[0554] Output: Updates to system logs, instructions to move autonomous mobile devices to standby positions

[0555] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0556] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0557] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0558] [Second embodiment]

[0559] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0560] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0561] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0562] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0563] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0564] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0565] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0566] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0567] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0568] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0569] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0570] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0571] This invention is a system that supports residents in moving from their homes safely and efficiently, and is realized by linking user terminals, servers, robots, and electric senior cars.

[0572] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[0573] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereinafter referred to as the robot). The server manages the user's location information and request content, and monitors the robot's location and status in real time.

[0574] The robot receives instructions from the server and heads to the designated entrance lobby, where it connects the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. AI technology is also used to enable smooth communication with residents. For example, the robot can notify the user of its arrival time in advance and provide directions along the way. The robot connects the senior car and heads to the user's entrance lobby.

[0575] The user gets into the senior car in the entrance lobby and travels with the robot to their room in their home. During this time, the robot uses sensors and AI technology to drive safely, and can also accept remote control via control signals sent from the server as needed. The user's device can check the robot's location in real time, allowing the user to travel with peace of mind.

[0576] As a concrete example, we will show a scenario in which an elderly resident, Mr. A, wants to return to his room at home after going out.

[0577] 1. User (A):

[0578] Open the smartphone app and enter your pickup request.

[0579] 2. Device (app):

[0580] Send A's request to the server.

[0581] 3. Server:

[0582] Upon receiving the request, confirm Mr. A's location and his room number at home.

[0583] It instructs the nearest robot to move to the entrance lobby and connect the senior cart.

[0584] 4. Robots:

[0585] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[0586] 5. Robot and Person A:

[0587] We welcome Mr. A in the entrance lobby and guide him to the senior car.

[0588] After checking for safety, I head to my room.

[0589] 6. User Device:

[0590] The robot's location is displayed in real time, providing Mr. A with a sense of security.

[0591] 7. Arrival:

[0592] We arrive in front of Mr. A's house and park the mobility scooter.

[0593] The robot notifies the server that the transfer is complete.

[0594] This system provides an environment where elderly people can move around the apartment building safely and efficiently, and is expected to promote social participation and maintain health. This invention is a practical means to improve the quality of life for elderly people and also brings benefits to apartment building managers.

[0595] The processing flow will be explained below.

[0596] Step 1:

[0597] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[0598] Step 2:

[0599] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[0600] Step 3:

[0601] The terminal transmits the pickup request input by the user to the server, including the user's location information and destination information.

[0602] Step 4:

[0603] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[0604] Step 5:

[0605] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[0606] Step 6:

[0607] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[0608] Step 7:

[0609] When the robot arrives at the entrance lobby, it will notify the user of its arrival via voice guidance.

[0610] Step 8:

[0611] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[0612] Step 9:

[0613] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[0614] Step 10:

[0615] The terminal transmits the user's "depart" instruction to the server.

[0616] Step 11:

[0617] The server sends a "go" command to the robot.

[0618] Step 12:

[0619] The robot receives the "depart" command and begins autonomously moving towards the room in the home. It uses its built-in sensors to detect obstacles and drive safely.

[0620] Step 13:

[0621] The terminal obtains the robot's location information in real time and provides it to the user.

[0622] Step 14:

[0623] When the robot arrives at its destination, it will notify the user by voice.

[0624] Step 15:

[0625] The user gets out of the mobility scooter and enters a room in their home.

[0626] Step 16:

[0627] The robot notifies the server that the transfer is complete.

[0628] Step 17:

[0629] The server marks the robot's task as completed and checks for the robot's next task.

[0630] Step 18:

[0631] The robot moves to an appropriate position to wait for the next instruction if there is no next task.

[0632] This series of processes will result in a system that will enable safe and secure transportation for the elderly.

[0633] Example 1

[0634] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0635] In modern society, there is a demand for support systems that enable elderly people and people with disabilities to travel efficiently and safely. However, current systems lack the technical capabilities to provide real-time location information, detect obstacles for safe driving, and provide autonomous mobility support. This makes it difficult to provide an environment where elderly people and people with disabilities can go out with peace of mind.

[0636] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0637] In this invention, the server includes a data analysis means for analyzing requests from the information processing device and issuing appropriate instructions, a mechanism means for the mechanical device to move in response to instructions and connect to the mobile device at a specified location, and a mechanical device that moves autonomously and connects to the mobile device based on instructions sent from the data processing device, thereby enabling elderly people and people with disabilities to move efficiently and safely.

[0638] "User" refers to a person who uses the system to receive mobility assistance.

[0639] The term "information processing device" refers to a terminal device operated by a user, which inputs and transmits requests.

[0640] "Data processing device" refers to a central system that receives and analyzes requests sent from an information processing device and issues instructions.

[0641] The term "mechanical device" refers to a device such as a robot that operates autonomously based on instructions from a data processing device and that supports the movement of a user by connecting to a mobile device.

[0642] "Mobility device" refers to a device such as an electric senior car or wheelchair that is connected to a mechanical device to transport a user.

[0643] "Sensor" refers to a device installed on a machine to detect obstacles and sense the surrounding environment.

[0644] "Data analysis means" refers to a method or technology for analyzing a request sent from an information processing device and creating appropriate instructions.

[0645] The term "mechanical means" refers to a mechanism by which a mechanical device connects a moving device based on an instruction.

[0646] "Artificial intelligence technology" refers to machine learning and natural language processing technologies used by machines to smoothly communicate with users.

[0647] This invention is a system that supports the elderly and physically disabled to move around safely and efficiently. The system consists of a user terminal, a data processing device (server), a mechanical device (robot), and a mobility device (electric senior car). The specific hardware and software configurations, as well as data processing and calculation methods, are explained below.

[0648] Hardware and software used

[0649] User terminal: Includes information processing devices such as smartphones and tablets, on which dedicated applications are installed.

[0650] Data Processing Unit (Server): The central system that analyzes requests and issues appropriate instructions to the robot. It includes a database and a communication interface.

[0651] Mechanical device (robot): A robot capable of autonomous movement, equipped with high-precision sensors and artificial intelligence (AI) technology.

[0652] Mobility device (electric senior car): A means of transporting the user in cooperation with the robot. It operates by being connected to the robot.

[0653] Program processing overview

[0654] User device operation

[0655] Users input and send their transportation requests using a dedicated application, which includes functions such as inputting requests, obtaining current locations, and displaying real-time location information.

[0656] Server Processing

[0657] The server analyzes requests received from the user's device and confirms the user's current location and destination. Based on the analysis results, it issues instructions to the nearest robot, including moving to the entrance lobby and connecting to an electric senior car. The server also monitors the robot's location and status in real time and sends control signals as necessary.

[0658] Robot movement

[0659] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves and connects to the electric senior car. Upon arrival, the robot greets the user and guides them to the senior car. During the journey, it makes full use of AI technology to check for safety and communicates smoothly with the user as needed.

[0660] Specific example explanation

[0661] When elderly resident A goes out and wants to return to his room, the following steps are taken:

[0662] 1. User: Opens the smartphone app and enters a pickup request.

[0663] Example prompt: How do I enter a destination using my smartphone?

[0664] 2. Terminal: The user terminal sends a request to the server.

[0665] Sample prompt: Describe the process a user goes through to submit a trip request in your app.

[0666] 3. Server: Receives the request, analyzes it, and sends instructions to the nearest robot.

[0667] Sample prompt: Please explain in detail how the server parses the request.

[0668] 4. Robot: Following instructions, it heads to the entrance lobby and connects the electric senior car.

[0669] Sample prompt: How can the robot navigate to the designated location and connect the mobility scooter?

[0670] 5. Robot: Welcomes the user in the entrance lobby and guides them to the mobility scooter.

[0671] Sample prompt: Please explain in detail the steps the robot takes to pick up the user.

[0672] This system provides an environment where elderly people and people with disabilities can move around safely and efficiently.

[0673] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0674] Step 1:

[0675] Enter your transportation request

[0676] User:

[0677] The resident opens the smartphone app, taps the "Request a Pickup" button on the app's home screen, and selects their room as the destination.

[0678] Input: Destination selection, user location information

[0679] Output: Pickup request information (user ID, current location, destination, timestamp)

[0680] Step 2:

[0681] Submit a Pickup Request

[0682] Device:

[0683] The user device sends A's request information to the server. When the user taps the send button, the request information is sent to the server via the network.

[0684] Input: Pickup request information

[0685] Output: HTTP request to the server

[0686] Step 3:

[0687] Receiving and parsing the request

[0688] server:

[0689] The server receives and analyzes the request sent from the user's device. The server stores the received request information (user ID, room number, current location) in a database and assigns the nearest robot.

[0690] Input: HTTP request (transport request information)

[0691] Output: Instructions to the robot (destination, current location, user information)

[0692] Step 4:

[0693] Robot instructions

[0694] server:

[0695] The server instructs the nearest robot to move to the entrance lobby and connect to the electric senior car. The server sends an HTTP request to the robot, instructing it to move and connect to the senior car. After the robot receives the command, it begins to act.

[0696] Input: Instructions for the robot

[0697] Output: Robot starts moving

[0698] Step 5:

[0699] Robot movement and connection

[0700] robot:

[0701] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves, and the mechanism for connecting the electric senior car to it is activated.

[0702] Input: Instructions from the server

[0703] Output: Arrival at the entrance lobby and docking of the mobility scooter

[0704] Step 6:

[0705] Meet and greet users

[0706] robot:

[0707] The robot greets Mr. A in the entrance lobby and guides him to the senior car. It calls out to Mr. A by voice and guides him to the boarding position of the senior car. The robot uses AI-based natural language processing to converse with him.

[0708] Input: User pickup instructions

[0709] Output: Directions to the user's mobility scooter

[0710] Step 7:

[0711] Providing real-time information

[0712] Device:

[0713] The user's device displays the robot's real-time location information. The app displays a map on the screen, and the current locations of the robot and the user are updated in real time, giving the user a sense of security.

[0714] Input: Robot location

[0715] Output: Real-time map display

[0716] Step 8:

[0717] Completing the move

[0718] robot:

[0719] The robot and the senior car deliver Mr. A to his room at home. After arriving, the robot notifies the server that the transfer is complete. The server receives this notification and records the completion of the transfer in the database.

[0720] Input: Arrival at destination

[0721] Output: Notification of completion of transfer to server

[0722] (Application example 1)

[0723] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0724] Existing technologies for supporting elderly and disabled residents to travel safely and efficiently from their homes to their destinations face challenges in terms of safety and convenience. Furthermore, there is a lack of ways for residents to visually check real-time information while traveling, making it difficult for them to travel safely. Furthermore, technology is needed to enable smooth communication between residents and robots.

[0725] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0726] In this invention, the server includes a user terminal operated by the resident to provide assistance with going out, a means for receiving and managing requests sent from the user terminal, a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, the electric senior car for transporting the resident while connected to the robot, a means for detecting obstacles using sensors mounted on the robot and safely driving it, a means for remotely controlling the robot's operation using control signals sent from the server, and a means for visualizing the robot's position information and movement route in real time using smart glasses or a head-mounted display. This enables residents to travel safely and efficiently from their homes to their destinations with peace of mind.

[0727] A "user terminal" is an electronic device operated by residents to assist them in going out, and includes smartphones, tablets, and the like.

[0728] A "server" is a central management device that receives and manages requests sent from user terminals.

[0729] A "robot" is a mechanical device that moves autonomously based on instructions sent from a server and connects to an electric senior car.

[0730] An "electric senior car" is an electric vehicle that is connected to a robot and used to transport residents.

[0731] A "sensor" is a device installed on a robot that detects obstacles.

[0732] A "control signal" is an operation command signal sent from the server to the robot.

[0733] "Remote control means" refers to a method of remotely controlling the robot's movements using control signals sent from a server.

[0734] "Smart glasses" are wearable devices that can visually display the robot's position information and movement route.

[0735] A "head-mounted display" is a device worn by a user on the head that displays information, and is used to visualize the position information and movement path of a robot.

[0736] "Real-time information" refers to data or information that provides an immediate picture of the current situation.

[0737] "AI technology" refers to artificial intelligence technology that is installed in robots and enables smooth communication with residents.

[0738] "Visualization means" refers to a method for visualizing the robot's position information and movement path using smart glasses, a head-mounted display, or the like.

[0739] This invention is a support system for elderly people and people with disabilities to move from their homes to their destinations safely and efficiently. A specific embodiment of the system is described below.

[0740] System configuration

[0741] The system includes a user terminal, a server, a robot, an electric senior car, smart glasses, and a head-mounted display.

[0742] 1. User Device

[0743] A user terminal is an electronic device, such as a smartphone or tablet, that a resident uses to input a transportation request.

[0744] The application installed on the user terminal has a function for sending requests and a function for displaying real-time location information.

[0745] 2. Server

[0746] The server receives and analyzes the request sent from the user terminal.

[0747] The server manages the user's location information and request details, and sends instructions to the most appropriate robot.

[0748] The server is built using cloud services such as AWS, and communication is carried out in real time using WebSocket.

[0749] 3. Robot

[0750] The autonomous mobile robot moves automatically and connects to the electric senior car based on instructions from the server.

[0751] The robot is equipped with high-precision sensors (e.g., LIDAR, cameras) to detect and avoid obstacles.

[0752] Using AI technology (e.g. TensorFlow), we communicate smoothly with residents and provide guidance along the way.

[0753] 4. Electric senior car

[0754] The electric senior car is an electric vehicle connected to a robot to transport residents.

[0755] Senior cars provide safe and comfortable transportation.

[0756] 5. Smart Glasses and Head-Mounted Displays

[0757] Smart glasses or head-mounted displays (e.g., Google Glass, Microsoft HoloLens) are used to visualize the robot's position and path.

[0758] This allows residents to check the robot's status in real time and move around with peace of mind.

[0759] Example

[0760] For example, when a resident moves from their room to the entrance of an apartment building, they follow the steps below.

[0761] 1. Request transmission from user terminal

[0762] The user starts the application on their device and inputs a pickup request, which includes destination information and current location.

[0763] 2. Request Processing on the Server

[0764] The server receives the request, analyzes it, and sends instructions to the most suitable robot.

[0765] 3. Robot movement and resident guidance

[0766] The robot receives instructions, starts moving autonomously, connects to the electric senior cart, and detects and avoids obstacles as necessary.

[0767] The robot will notify the user's device or smart glasses of its arrival time and provide guidance along the way.

[0768] 4. Displaying real-time information

[0769] The robot's position and movement path are visualized through smart glasses or a head-mounted display.

[0770] Prompt example

[0771] Possible input prompts for a generative AI model include:

[0772] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[0773] This system allows residents to travel from their homes to their destinations efficiently and safely with peace of mind. This completes the embodiment of the present invention.

[0774] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0775] Step 1:

[0776] The application is started on the user terminal and a transportation request is input.

[0777] The input includes current location and destination information.

[0778] The user terminal transmits the input data to the server.

[0779] Input: Current location and destination information from the user

[0780] Output: Sending request data to the server

[0781] Step 2:

[0782] The server receives the request sent from the user terminal.

[0783] The server analyzes the request data and obtains the user's current location and destination information.

[0784] Input: Request data sent from the user terminal

[0785] Output: Parsed current location and destination information

[0786] Step 3:

[0787] The server checks the current location of the robots to select the most suitable robot.

[0788] A movement instruction is generated for the optimal robot and sent to the robot.

[0789] Input: Current location and destination information, and robot location data

[0790] Output: Movement instructions to the robot

[0791] Step 4:

[0792] The robot receives movement instructions from the server and begins moving autonomously.

[0793] The robot connects to an electric senior car and moves towards its destination.

[0794] Input: Move instruction from server

[0795] Output: Robot starts moving

[0796] Step 5:

[0797] Sensors installed on the robot detect obstacles and select a safe route for driving.

[0798] It processes sensor data in real time and performs data calculations to avoid obstacles.

[0799] Input: Sensor data from the surrounding environment

[0800] Output: Path information for obstacle avoidance

[0801] Step 6:

[0802] The robot sends its location information and estimated arrival time to the server.

[0803] The server provides this information to user devices, smart glasses, and head-mounted displays in real time.

[0804] Input: Robot location and estimated arrival time

[0805] Output: Real-time notifications to user devices and smart glasses

[0806] Step 7:

[0807] The user visually checks the robot's location and movement route using smart glasses or a head-mounted display.

[0808] This allows users to travel with peace of mind.

[0809] Input: Location information and movement route data from the server

[0810] Output: Real-time display as visual information

[0811] Step 8:

[0812] When the robot arrives at its destination, it sends an arrival notification to the server.

[0813] The server notifies the user terminal of the arrival information, and the series of movements is completed.

[0814] Input: Arrival notification from the robot

[0815] Output: Arrival information notification to user terminal

[0816] Prompt example

[0817] Possible input prompts for a generative AI model include:

[0818] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[0819] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0820] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions, and is realized by linking the user terminal, server, robot, electric senior car, and emotion engine.

[0821] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[0822] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereafter referred to as the robot) based on the user's location and destination information. The server monitors the robot's location and status in real time, and sends control signals as necessary to remotely control the robot's operation.

[0823] The robot receives instructions from the server and heads to the designated entrance lobby to connect the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. It is also capable of smooth communication with residents using AI technology. It is also equipped with an emotion engine that can analyze the user's emotional state and respond appropriately. The emotion engine recognizes emotions based on the user's facial expressions, voice, and behavioral data, and the robot responds appropriately.

[0824] The specific operation of the system will be described below.

[0825] Consider a scenario where Resident A goes out and wants to return to his room at home.

[0826] 1. User (A):

[0827] Open the smartphone app and enter your pickup request.

[0828] 2. Device (app):

[0829] A request for transportation from Mr. A is sent to the server.

[0830] 3. Server:

[0831] Receive the request and confirm Mr. A's location and destination.

[0832] It instructs the nearest robot to move and perform a series of actions.

[0833] 4. Robots:

[0834] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[0835] The system uses an emotion engine to analyze Mr. A's emotional state and greets him appropriately upon arrival.

[0836] 5. User (A):

[0837] Check the senior car connected to the robot in the entrance lobby and get in.

[0838] Travel with peace of mind thanks to the robot's voice guidance.

[0839] 6. Robots:

[0840] It uses sensors to avoid obstacles and safely drive towards your room in the home.

[0841] If Mr. A seems anxious while driving, the emotion engine will detect this and reassure him with appropriate words.

[0842] 7. Terminal:

[0843] The robot's location is displayed in real time and provided to Mr. A.

[0844] 8. Robots:

[0845] When they arrive at their destination, they inform Mr. A of their arrival and greet him in a friendly manner.

[0846] 9. User (A):

[0847] I get off the senior car and enter my room at home.

[0848] 10. Robots:

[0849] The server is notified that the transfer is complete, and the emotional state of Mr. A recognized by the emotion engine is also sent.

[0850] If there is no next task, move to an appropriate waiting position.

[0851] This series of processes will realize a system that allows users to move around the apartment building safely and efficiently while taking their emotions into consideration. The emotion engine's response is expected to provide an elderly-friendly environment and improve the quality of life.

[0852] The processing flow will be explained below.

[0853] Step 1:

[0854] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[0855] Step 2:

[0856] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[0857] Step 3:

[0858] The device transmits the location information along with the pickup request entered by the user to the server, including the user's destination information.

[0859] Step 4:

[0860] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[0861] Step 5:

[0862] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[0863] Step 6:

[0864] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[0865] Step 7:

[0866] When the robot arrives at the entrance lobby, it notifies the user of its arrival with a voice prompt. It also uses a built-in emotion engine to analyze the user's emotions and provides voice prompts and greetings according to the user's emotional state.

[0867] Step 8:

[0868] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[0869] Step 9:

[0870] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[0871] Step 10:

[0872] The terminal transmits the user's "depart" instruction to the server.

[0873] Step 11:

[0874] The server sends a "go" command to the robot.

[0875] Step 12:

[0876] The robot receives the "depart" command and begins autonomously moving toward the room in the home. It uses built-in sensors to detect obstacles and drive safely. It also uses an emotion engine to monitor the user's emotions.

[0877] Step 13:

[0878] The robot analyzes the user's emotions while moving, and if it detects anxiety or tension, it will provide voice guidance and dialogue to provide comfort.

[0879] Step 14:

[0880] The terminal displays the robot's location information in real time and provides it to the user.

[0881] Step 15:

[0882] When the robot arrives at its destination, it notifies the user and speaks in a friendly manner. Based on the analysis results of its emotion engine, it checks whether the user is calm.

[0883] Step 16:

[0884] The user gets out of the mobility scooter and enters a room in their home.

[0885] Step 17:

[0886] The robot notifies the server that the transfer is complete and also sends the user's emotional state as recognized by the emotion engine.

[0887] Step 18:

[0888] The server marks the robot's task as completed and checks to see if there are any other tasks waiting.

[0889] Step 19:

[0890] If there is no next task, the robot moves to an appropriate waiting position and waits for the next instruction.

[0891] This series of processes will realize a system that utilizes an emotion engine to allow elderly people to move safely around an apartment building while also taking into consideration their emotional state.

[0892] Example 2

[0893] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0894] Conventional outing support systems simply support movement without considering the user's emotional state. As a result, elderly and emotionally unstable users often feel anxious while out and about. Another issue is that communication with the user is not smooth, making it difficult for the user to feel familiar with the system.

[0895] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a user terminal operated by the resident to provide outing assistance, means for receiving and managing requests sent from the user terminal, means including a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, an electric senior car for transporting the resident while connected to the robot, means for detecting obstacles using sensors mounted on the robot and driving safely, means for remotely controlling the operation of the robot using control signals sent from the server, means for analyzing the emotional state of the resident using an emotion engine, and means for providing appropriate greetings or calls based on the user's emotions. This enables safe and friendly outing assistance that takes the user's emotional state into consideration.

[0896] A "user terminal" is an electronic device operated by a user, and includes a smartphone, a tablet, and the like.

[0897] A "server" is a computer system that receives and manages requests sent from user terminals.

[0898] A "request" is request information sent from a user terminal to a server, and includes instructions for transportation or movement.

[0899] The "robot" is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car.

[0900] An "electric senior car" is an electric mobility device that is connected to a robot to safely transport residents.

[0901] "Sensors" are devices installed on robots to detect obstacles, and include LIDAR and cameras.

[0902] A "control signal" is an instruction sent from the server to the robot to remotely control the robot's operations.

[0903] An "emotion engine" is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response.

[0904] "Appropriate greetings and calls" are communication actions that the robot takes toward the user based on the analysis results of the emotion engine.

[0905] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions. The main components of the system are a user terminal, a server, a robot, an electric senior car, and an emotion engine, all of which work in conjunction with each other.

[0906] User terminal

[0907] A user terminal is an electronic device operated by a user, and includes smartphones, tablets, etc. A dedicated application is installed on the device, and the user uses this application to input a pickup request. For example, simply inputting a request such as "I want to go to my room at home" initiates a series of operations in the system.

[0908] server

[0909] The server is a computer system that receives and manages requests sent from the user's terminal. The server analyzes the request data and sends appropriate instructions to the robot based on the user's current location and destination. During this process, the server constantly monitors the robot's location and status in real time, and sends control signals each time to remotely control the robot's operation.

[0910] robot

[0911] The robot is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car. The robot is equipped with high-precision sensors and can move safely while detecting obstacles using LIDAR and cameras. The robot also has an emotion engine that recognizes emotions based on the user's facial expressions, voice, and behavioral data. For example, if the user appears anxious, the robot can reassure them with appropriate words such as, "Don't worry. We'll soon arrive at our destination."

[0912] Electric senior car

[0913] An electric senior car is an electric mobility device that connects to a robot to safely transport residents. Because it is controlled by the robot, the user does not need to drive it themselves. It allows residents to travel safely and smoothly to specific destinations within the apartment complex.

[0914] Emotion Engine

[0915] The emotion engine is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response. The emotion engine uses data obtained from a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing their emotional state in real time. This allows the robot to respond appropriately to the user, providing a sense of security, especially for elderly and emotionally unstable users.

[0916] Add specific examples to the description

[0917] For example, if resident A wants to return to their room after being out, the following scenario can be considered: First, resident A opens the smartphone app and enters a request for transportation. The app sends the request to the server, which receives and analyzes the request, and then sends instructions to the nearest robot. Based on instructions from the server, the robot heads to the entrance lobby and connects its electric senior car. The robot then uses its emotion engine to analyze resident A's emotional state and responds with an appropriate greeting, such as, "Hello, resident A. How are you feeling today?" resident A gets into the senior car and follows the robot's guidance, feeling reassured as they travel. Finally, when the robot arrives at its destination and resident A gets out of the senior car, the robot notifies the server that transportation is complete and waits for the next task.

[0918] Prompt Sentence Examples

[0919] Below are some example prompts for a generative AI model:

[0920] "Please explain how an autonomous mobile robot equipped with an emotion engine can analyze the emotional state of its user."

[0921] "Please explain in detail the procedure for welcoming a robot using the emotion engine."

[0922] This not only allows users to use the system with peace of mind, but also provides an environment that is particularly friendly to elderly people.The present invention aims to improve the quality of life of users.

[0923] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0924] Step 1:

[0925] The user opens a dedicated app installed on their smartphone or tablet and enters a transportation request.

[0926] Specific operation: The user launches the app, enters their destination (e.g., their room number at home), and presses the pick-up button.

[0927] Input: Trip request information entered by the user (destination, current location).

[0928] Output: Trip request data generated within the app.

[0929] Step 2:

[0930] The terminal sends the user's transportation request to the server.

[0931] Specific behavior: The app sends a request containing the user's location information (GPS data) and destination information to the server as an HTTP POST request.

[0932] Input: The generated trip request data.

[0933] Output: The request data sent to the server.

[0934] Step 3:

[0935] The server receives the request and verifies the user's location and destination.

[0936] Specific operation: The server analyzes the request data and identifies the user's current location (for example, the entrance to an apartment building).

[0937] Input: Request data sent from the device.

[0938] Output: Parsed user location and destination information.

[0939] Step 4:

[0940] The server instructs the nearest robot to move and perform a series of actions.

[0941] Specific operation: The server checks the robot's current location, calculates the optimal route, and then sends instructions such as "head to the entrance, connect the senior cart, and welcome the user."

[0942] Input: User's current location, destination, and robot's current location data.

[0943] Output: Movement and motion instruction data sent to the robot.

[0944] Step 5:

[0945] The robot receives instructions from the server, heads to the entrance lobby, and connects the electric senior car.

[0946] Specific operation: The robot uses its own navigation system to autonomously navigate to the entrance lobby and physically connect with the senior car.

[0947] Input: Movement and action instruction data sent from the server.

[0948] Output: Arrival at the entrance lobby and the connection status of the mobility scooter.

[0949] Step 6:

[0950] The robot uses an emotion engine to analyze the user's emotional state and greets them appropriately upon arrival.

[0951] Specific operation: The robot uses a camera and microphone to analyze the user's facial expressions and tone of voice, and then greets them with a greeting such as, "Hello, Mr. A. How are you feeling today?"

[0952] Input: User's facial expression data, voice data.

[0953] Output: Analysis results and a voice message based on them.

[0954] Step 7:

[0955] The user checks the senior car connected to the robot in the entrance lobby and gets in.

[0956] Specific operation: The user follows the robot's instructions to get into the senior car.

[0957] Input: User confirmation and ride operation.

[0958] Output: Riding in a mobility scooter.

[0959] Step 8:

[0960] The robot's voice guidance provides users with a sense of security as they move around.

[0961] What it does: The robot provides voice guidance such as, "Don't worry, we're on our way to your next destination."

[0962] Input: Robot voice guidance data.

[0963] Output: A voice message to the user.

[0964] Step 9:

[0965] The robot uses sensors to avoid obstacles and safely navigates towards your room.

[0966] How it works: The robot uses a LIDAR sensor to scan the surrounding environment and automatically selects a safe route to move.

[0967] Input: Sensor data, environmental data.

[0968] Output: Obstacle-avoiding route information and safe movement status.

[0969] Step 10:

[0970] If the user appears anxious while the robot is driving, the emotion engine will detect this and provide appropriate reassurance.

[0971] What it does: The emotion engine detects the user's anxiety and provides reassuring words such as, "You're almost there."

[0972] Input: User's facial expression data, voice data.

[0973] Output: Analysis results and a voice message based on them.

[0974] Step 11:

[0975] The robot arrives at the destination and notifies the user of its arrival while speaking in a friendly manner.

[0976] Specific operation: The robot will provide guidance such as, "Mr. A, you have arrived at your destination. Thank you for your hard work."

[0977] Input: Destination arrival data.

[0978] Output: A voice message to the user.

[0979] Step 12:

[0980] The user gets out of the mobility scooter and enters a room in their home.

[0981] Specific actions: The user gets out of the mobility scooter, unlocks the door to their home, and enters.

[0982] Input: User exit and entry operations.

[0983] Output: The user's room entry status.

[0984] Step 13:

[0985] The robot notifies the server that the transfer is complete and also transmits the user's emotional state as recognized by the emotion engine.

[0986] Specific operation: The robot uploads the data of the completed transfer to the server and also sends the emotion data.

[0987] Input: Pick-up and drop-off completion data, emotional state data.

[0988] Output: Data sent to the server.

[0989] Step 14:

[0990] If the robot has no next task, it moves to an appropriate waiting position.

[0991] Specific behavior: The robot moves to the waiting station and waits for further instructions.

[0992] Input: Next task availability data.

[0993] Output: Waiting in standby position.

[0994] (Application example 2)

[0995] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0996] There is a need for systems that allow elderly people and people who require physical assistance to shop safely and with peace of mind in physical stores. It is also important to provide appropriate support according to their emotional state. Existing systems have had problems with insufficient emotion recognition and are unable to adequately respond to the anxiety and confusion of elderly people.

[0997] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and managing requests sent from a user terminal, means for controlling an autonomous mobile device that moves autonomously based on instructions sent from the server and connects to an electric vehicle, and means for recognizing the emotional state of residents using an emotion engine installed in the autonomous mobile device and taking appropriate action. This enables elderly people and people who require physical assistance to move safely within a store and shop with peace of mind. Furthermore, the emotion engine provides appropriate support according to the user's emotional state, thereby improving their sense of security and satisfaction.

[0998] "Residents" are users of the system, primarily elderly people and those who require physical assistance.

[0999] "User terminal" refers to electronic devices such as smartphones and tablets that are operated by residents.

[1000] A "request" refers to information sent from a user terminal to a server to request support or movement.

[1001] The term "server" refers to a computer system that manages requests sent from user terminals and sends instructions to autonomous mobile devices.

[1002] "Autonomous mobile equipment" refers to robots and similar equipment that move autonomously and connect to electric vehicles based on instructions from a server to support residents.

[1003] "Electric vehicle" refers to an electric senior car or other electrically powered vehicle that is connected to an autonomous mobility device and used to transport residents.

[1004] "Sensor" refers to a device installed on an autonomous mobile device to detect obstacles in the surrounding area.

[1005] "Emotion engine" refers to artificial intelligence technology that recognizes residents' emotional state and responds appropriately.

[1006] "Artificial intelligence technology" refers to technology used by autonomous mobile devices to facilitate communication with residents.

[1007] This invention is a system that helps elderly people and people who require physical assistance to shop safely in physical stores. The system is composed of components including a server, user terminals, autonomous mobile devices, electric vehicles, sensors, an emotion engine, and artificial intelligence technology.

[1008] First, the user device is a smartphone or tablet. Residents submit support requests through a dedicated application. This request is sent from the device to a server. The server receives the request and analyzes the resident's location and the type of support required. Based on the analysis results, it sends instructions to the nearest autonomous mobile device.

[1009] The autonomous mobile device receives instructions from the server and moves to a designated location within the store. This device connects to an electric vehicle (such as an electric cart or senior cart) to transport the resident. The autonomous mobile device is equipped with sensors that detect and avoid obstacles in the surrounding area. It is also equipped with an emotion engine that analyzes the resident's facial expressions, voice, and behavior to recognize their emotional state and respond appropriately.

[1010] The server monitors the location of the autonomous mobile device in real time and remotely controls it as needed, using a high-precision control system.

[1011] As a specific example, consider the following scenario.

[1012] Scenario: Elderly person B goes shopping at a supermarket

[1013] 1. User (Mr. B):

[1014] Open the smartphone app, select "Shopping Support," and submit a support request.

[1015] 2. Device (app):

[1016] Send Mr. B's shopping support request to the server.

[1017] 3. Server:

[1018] Receive the request and confirm Mr. B's location and the support he needs.

[1019] Send instructions to the nearest autonomous mobile device.

[1020] 4. Autonomous mobile devices:

[1021] Following the server's instructions, I headed towards the entrance of the supermarket.

[1022] The emotional engine is used to analyze Mr. B's emotional state and provide an appropriate greeting or greeting upon arrival.

[1023] 5. Person B:

[1024] The user contacts an autonomous mobile device near the entrance and requests assistance with shopping.

[1025] Follow the voice guidance of the autonomous mobile device and start shopping with peace of mind.

[1026] 6. Autonomous mobile devices:

[1027] As they move around the store together, they guide B to products based on his shopping list.

[1028] Move safely while detecting obstacles.

[1029] If Person B feels anxious, the emotion engine will detect this and provide appropriate words to reassure them.

[1030] Hardware and Software Used

[1031] User device: Smartphone (iOS / Android)

[1032] Server: Cloud server (AWS or Google Cloud) + Flask (Python web server framework)

[1033] Autonomous mobile device: Autonomous mobile robot (equipped with LiDAR sensor, camera, and microphone)

[1034] Emotion Engine: Emotion Recognition Model with TensorFlow / Keras

[1035] An example prompt is:

[1036] "Imagine that Person B goes to the supermarket and asks an autonomous mobile device to help him with his shopping. Person B is feeling a little anxious. How would the autonomous mobile device respond to reassure Person B?"

[1037] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1038] Step 1:

[1039] User (Mr. B)

[1040] Open the smartphone app, select "Shopping Support," and submit a support request.

[1041] Input: B's location information, request details

[1042] Output: Support request data

[1043] Step 2:

[1044] Device (app)

[1045] Send Mr. B's shopping support request to the server.

[1046] Input: Support request data

[1047] Output: Request data sent to the server

[1048] Step 3:

[1049] server

[1050] The system receives the request, checks B's location and the support required, analyzes the request data, and sends instructions to the nearest autonomous mobile device.

[1051] Input: Request data sent to the server

[1052] Data processing / calculation: Analyzes the request and selects the most suitable autonomous mobile device

[1053] Output: Instruction data sent to the autonomous mobile device

[1054] Step 4:

[1055] Autonomous mobile equipment

[1056] It receives instructions from the server and moves to a designated location within the store. During its movement, it uses sensors to detect surrounding obstacles and proceed safely.

[1057] Input: Instruction data sent to the autonomous mobile device

[1058] Data processing / calculation: Calculating movement routes based on location information, detecting obstacles using sensors

[1059] Output: Action instructions to move safely

[1060] Step 5:

[1061] Autonomous mobile equipment

[1062] Upon arrival, the emotion engine is used to analyze Mr. B's emotional state and an appropriate greeting or greeting is given.

[1063] Input: Person B's facial expression and voice data

[1064] Data processing / calculation: Emotion recognition using TensorFlow / Keras, and decision on how to respond based on the analysis results

[1065] Output: An appropriate response to Person B

[1066] Step 6:

[1067] User (Mr. B)

[1068] Users contact an autonomous mobile device near the entrance and request shopping assistance. They then follow the audio guidance provided by the autonomous mobile device and begin shopping with peace of mind.

[1069] Input: Autonomous mobile device information

[1070] Output: Person B's actions

[1071] Step 7:

[1072] Autonomous mobile equipment

[1073] As the robot moves around the store together with B, it guides him to products based on his shopping list. Along the way, it again uses sensors to detect and avoid obstacles. If B feels confused or anxious, the emotion engine detects this and uses appropriate words to reassure him.

[1074] Input: B's shopping list, facial expressions, voice data, sensor information

[1075] Data processing / calculation: Navigation to product locations based on shopping lists, emotion recognition, obstacle avoidance

[1076] Output: Product guidance, obstacle avoidance behavior, appropriate response

[1077] Step 8:

[1078] Autonomous mobile equipment

[1079] Once the customer has finished shopping, they will be guided to the cash register and supported until payment is complete.

[1080] Input: B's actions, cash register location information

[1081] Data processing / calculation: Secure route calculation, payment completion confirmation

[1082] Output: Instructions to cashier, necessary responses

[1083] Step 9:

[1084] server

[1085] It notifies the robot that the shopping support is complete and records the emotional state of the robot according to the emotion engine. If there is no next task, the robot moves to an appropriate waiting position.

[1086] Input: Completion notification from autonomous mobile device, emotional state data

[1087] Data processing / calculation: Confirmation of task completion, movement plan to waiting position

[1088] Output: Updates to system logs, instructions to move autonomous mobile devices to standby positions

[1089] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1090] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1091] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1092] [Third embodiment]

[1093] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1094] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1095] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1096] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1097] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1098] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1099] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1100] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1101] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1102] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1103] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1104] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1105] This invention is a system that supports residents in moving from their homes safely and efficiently, and is realized by linking user terminals, servers, robots, and electric senior cars.

[1106] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[1107] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereinafter referred to as the robot). The server manages the user's location information and request content, and monitors the robot's location and status in real time.

[1108] The robot receives instructions from the server and heads to the designated entrance lobby, where it connects the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. AI technology is also used to enable smooth communication with residents. For example, the robot can notify the user of its arrival time in advance and provide directions along the way. The robot connects the senior car and heads to the user's entrance lobby.

[1109] The user gets into the senior car in the entrance lobby and travels with the robot to their room in their home. During this time, the robot uses sensors and AI technology to drive safely, and can also accept remote control via control signals sent from the server as needed. The user's device can check the robot's location in real time, allowing the user to travel with peace of mind.

[1110] As a concrete example, we will show a scenario in which an elderly resident, Mr. A, wants to return to his room at home after going out.

[1111] 1. User (A):

[1112] Open the smartphone app and enter your pickup request.

[1113] 2. Device (app):

[1114] Send A's request to the server.

[1115] 3. Server:

[1116] Upon receiving the request, confirm Mr. A's location and his room number at home.

[1117] It instructs the nearest robot to move to the entrance lobby and connect the senior cart.

[1118] 4. Robots:

[1119] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[1120] 5. Robot and Person A:

[1121] We welcome Mr. A in the entrance lobby and guide him to the senior car.

[1122] After checking for safety, I head to my room.

[1123] 6. User Device:

[1124] The robot's location is displayed in real time, providing Mr. A with a sense of security.

[1125] 7. Arrival:

[1126] We arrive in front of Mr. A's house and park the mobility scooter.

[1127] The robot notifies the server that the transfer is complete.

[1128] This system provides an environment where elderly people can move around the apartment building safely and efficiently, and is expected to promote social participation and maintain health. This invention is a practical means to improve the quality of life for elderly people and also brings benefits to apartment building managers.

[1129] The processing flow will be explained below.

[1130] Step 1:

[1131] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[1132] Step 2:

[1133] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[1134] Step 3:

[1135] The terminal transmits the pickup request input by the user to the server, including the user's location information and destination information.

[1136] Step 4:

[1137] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[1138] Step 5:

[1139] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[1140] Step 6:

[1141] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[1142] Step 7:

[1143] When the robot arrives at the entrance lobby, it will notify the user of its arrival via voice guidance.

[1144] Step 8:

[1145] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[1146] Step 9:

[1147] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[1148] Step 10:

[1149] The terminal transmits the user's "depart" instruction to the server.

[1150] Step 11:

[1151] The server sends a "go" command to the robot.

[1152] Step 12:

[1153] The robot receives the "depart" command and begins autonomously moving towards the room in the home. It uses its built-in sensors to detect obstacles and drive safely.

[1154] Step 13:

[1155] The terminal obtains the robot's location information in real time and provides it to the user.

[1156] Step 14:

[1157] When the robot arrives at its destination, it will notify the user by voice.

[1158] Step 15:

[1159] The user gets out of the mobility scooter and enters a room in their home.

[1160] Step 16:

[1161] The robot notifies the server that the transfer is complete.

[1162] Step 17:

[1163] The server marks the robot's task as completed and checks for the robot's next task.

[1164] Step 18:

[1165] The robot moves to an appropriate position to wait for the next instruction if there is no next task.

[1166] This series of processes will result in a system that will enable safe and secure transportation for the elderly.

[1167] Example 1

[1168] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1169] In modern society, there is a demand for support systems that enable elderly people and people with disabilities to travel efficiently and safely. However, current systems lack the technical capabilities to provide real-time location information, detect obstacles for safe driving, and provide autonomous mobility support. This makes it difficult to provide an environment where elderly people and people with disabilities can go out with peace of mind.

[1170] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1171] In this invention, the server includes a data analysis means for analyzing requests from the information processing device and issuing appropriate instructions, a mechanism means for the mechanical device to move in response to instructions and connect to the mobile device at a specified location, and a mechanical device that moves autonomously and connects to the mobile device based on instructions sent from the data processing device, thereby enabling elderly people and people with disabilities to move efficiently and safely.

[1172] "User" refers to a person who uses the system to receive mobility assistance.

[1173] The term "information processing device" refers to a terminal device operated by a user, which inputs and transmits requests.

[1174] "Data processing device" refers to a central system that receives and analyzes requests sent from an information processing device and issues instructions.

[1175] The term "mechanical device" refers to a device such as a robot that operates autonomously based on instructions from a data processing device and that supports the movement of a user by connecting to a mobile device.

[1176] "Mobility device" refers to a device such as an electric senior car or wheelchair that is connected to a mechanical device to transport a user.

[1177] "Sensor" refers to a device installed on a machine to detect obstacles and sense the surrounding environment.

[1178] "Data analysis means" refers to a method or technology for analyzing a request sent from an information processing device and creating appropriate instructions.

[1179] The term "mechanical means" refers to a mechanism by which a mechanical device connects a moving device based on an instruction.

[1180] "Artificial intelligence technology" refers to machine learning and natural language processing technologies used by machines to smoothly communicate with users.

[1181] This invention is a system that supports the elderly and physically disabled to move around safely and efficiently. The system consists of a user terminal, a data processing device (server), a mechanical device (robot), and a mobility device (electric senior car). The specific hardware and software configurations, as well as data processing and calculation methods, are explained below.

[1182] Hardware and software used

[1183] User terminal: Includes information processing devices such as smartphones and tablets, on which dedicated applications are installed.

[1184] Data Processing Unit (Server): The central system that analyzes requests and issues appropriate instructions to the robot. It includes a database and a communication interface.

[1185] Mechanical device (robot): A robot capable of autonomous movement, equipped with high-precision sensors and artificial intelligence (AI) technology.

[1186] Mobility device (electric senior car): A means of transporting the user in cooperation with the robot. It operates by being connected to the robot.

[1187] Program processing overview

[1188] User device operation

[1189] Users input and send their transportation requests using a dedicated application, which includes functions such as inputting requests, obtaining current locations, and displaying real-time location information.

[1190] Server Processing

[1191] The server analyzes requests received from the user's device and confirms the user's current location and destination. Based on the analysis results, it issues instructions to the nearest robot, including moving to the entrance lobby and connecting to an electric senior car. The server also monitors the robot's location and status in real time and sends control signals as necessary.

[1192] Robot movement

[1193] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves and connects to the electric senior car. Upon arrival, the robot greets the user and guides them to the senior car. During the journey, it makes full use of AI technology to check for safety and communicates smoothly with the user as needed.

[1194] Specific example explanation

[1195] When elderly resident A goes out and wants to return to his room, the following steps are taken:

[1196] 1. User: Opens the smartphone app and enters a pickup request.

[1197] Example prompt: How do I enter a destination using my smartphone?

[1198] 2. Terminal: The user terminal sends a request to the server.

[1199] Sample prompt: Describe the process a user goes through to submit a trip request in your app.

[1200] 3. Server: Receives the request, analyzes it, and sends instructions to the nearest robot.

[1201] Sample prompt: Please explain in detail how the server parses the request.

[1202] 4. Robot: Following instructions, it heads to the entrance lobby and connects the electric senior car.

[1203] Sample prompt: How can the robot navigate to the designated location and connect the mobility scooter?

[1204] 5. Robot: Welcomes the user in the entrance lobby and guides them to the mobility scooter.

[1205] Sample prompt: Please explain in detail the steps the robot takes to pick up the user.

[1206] This system provides an environment where elderly people and people with disabilities can move around safely and efficiently.

[1207] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1208] Step 1:

[1209] Enter your transportation request

[1210] User:

[1211] The resident opens the smartphone app, taps the "Request a Pickup" button on the app's home screen, and selects their room as the destination.

[1212] Input: Destination selection, user location information

[1213] Output: Pickup request information (user ID, current location, destination, timestamp)

[1214] Step 2:

[1215] Submit a Pickup Request

[1216] Device:

[1217] The user device sends A's request information to the server. When the user taps the send button, the request information is sent to the server via the network.

[1218] Input: Pickup request information

[1219] Output: HTTP request to the server

[1220] Step 3:

[1221] Receiving and parsing the request

[1222] server:

[1223] The server receives and analyzes the request sent from the user's device. The server stores the received request information (user ID, room number, current location) in a database and assigns the nearest robot.

[1224] Input: HTTP request (transport request information)

[1225] Output: Instructions to the robot (destination, current location, user information)

[1226] Step 4:

[1227] Robot instructions

[1228] server:

[1229] The server instructs the nearest robot to move to the entrance lobby and connect to the electric senior car. The server sends an HTTP request to the robot, instructing it to move and connect to the senior car. After the robot receives the command, it begins to act.

[1230] Input: Instructions for the robot

[1231] Output: Robot starts moving

[1232] Step 5:

[1233] Robot movement and connection

[1234] robot:

[1235] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves, and the mechanism for connecting the electric senior car to it is activated.

[1236] Input: Instructions from the server

[1237] Output: Arrival at the entrance lobby and docking of the mobility scooter

[1238] Step 6:

[1239] Meet and greet users

[1240] robot:

[1241] The robot greets Mr. A in the entrance lobby and guides him to the senior car. It calls out to Mr. A by voice and guides him to the boarding position of the senior car. The robot uses AI-based natural language processing to converse with him.

[1242] Input: User pickup instructions

[1243] Output: Directions to the user's mobility scooter

[1244] Step 7:

[1245] Providing real-time information

[1246] Device:

[1247] The user's device displays the robot's real-time location information. The app displays a map on the screen, and the current locations of the robot and the user are updated in real time, giving the user a sense of security.

[1248] Input: Robot location

[1249] Output: Real-time map display

[1250] Step 8:

[1251] Completing the move

[1252] robot:

[1253] The robot and the senior car deliver Mr. A to his room at home. After arriving, the robot notifies the server that the transfer is complete. The server receives this notification and records the completion of the transfer in the database.

[1254] Input: Arrival at destination

[1255] Output: Notification of completion of transfer to server

[1256] (Application example 1)

[1257] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1258] Existing technologies for supporting elderly and disabled residents to travel safely and efficiently from their homes to their destinations face challenges in terms of safety and convenience. Furthermore, there is a lack of ways for residents to visually check real-time information while traveling, making it difficult for them to travel safely. Furthermore, technology is needed to enable smooth communication between residents and robots.

[1259] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1260] In this invention, the server includes a user terminal operated by the resident to provide assistance with going out, a means for receiving and managing requests sent from the user terminal, a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, the electric senior car for transporting the resident while connected to the robot, a means for detecting obstacles using sensors mounted on the robot and safely driving it, a means for remotely controlling the robot's operation using control signals sent from the server, and a means for visualizing the robot's position information and movement route in real time using smart glasses or a head-mounted display. This enables residents to travel safely and efficiently from their homes to their destinations with peace of mind.

[1261] A "user terminal" is an electronic device operated by residents to assist them in going out, and includes smartphones, tablets, and the like.

[1262] A "server" is a central management device that receives and manages requests sent from user terminals.

[1263] A "robot" is a mechanical device that moves autonomously based on instructions sent from a server and connects to an electric senior car.

[1264] An "electric senior car" is an electric vehicle that is connected to a robot and used to transport residents.

[1265] A "sensor" is a device installed on a robot that detects obstacles.

[1266] A "control signal" is an operation command signal sent from the server to the robot.

[1267] "Remote control means" refers to a method of remotely controlling the robot's movements using control signals sent from a server.

[1268] "Smart glasses" are wearable devices that can visually display the robot's position information and movement route.

[1269] A "head-mounted display" is a device worn by a user on the head that displays information, and is used to visualize the position information and movement path of a robot.

[1270] "Real-time information" refers to data or information that provides an immediate picture of the current situation.

[1271] "AI technology" refers to artificial intelligence technology that is installed in robots and enables smooth communication with residents.

[1272] "Visualization means" refers to a method for visualizing the robot's position information and movement path using smart glasses, a head-mounted display, or the like.

[1273] This invention is a support system for elderly people and people with disabilities to move from their homes to their destinations safely and efficiently. A specific embodiment of the system is described below.

[1274] System configuration

[1275] The system includes a user terminal, a server, a robot, an electric senior car, smart glasses, and a head-mounted display.

[1276] 1. User Device

[1277] A user terminal is an electronic device, such as a smartphone or tablet, that a resident uses to input a transportation request.

[1278] The application installed on the user terminal has a function for sending requests and a function for displaying real-time location information.

[1279] 2. Server

[1280] The server receives and analyzes the request sent from the user terminal.

[1281] The server manages the user's location information and request details, and sends instructions to the most appropriate robot.

[1282] The server is built using cloud services such as AWS, and communication is carried out in real time using WebSocket.

[1283] 3. Robot

[1284] The autonomous mobile robot moves automatically and connects to the electric senior car based on instructions from the server.

[1285] The robot is equipped with high-precision sensors (e.g., LIDAR, cameras) to detect and avoid obstacles.

[1286] Using AI technology (e.g. TensorFlow), we communicate smoothly with residents and provide guidance along the way.

[1287] 4. Electric senior car

[1288] The electric senior car is an electric vehicle connected to a robot to transport residents.

[1289] Senior cars provide safe and comfortable transportation.

[1290] 5. Smart Glasses and Head-Mounted Displays

[1291] Smart glasses or head-mounted displays (e.g., Google Glass, Microsoft HoloLens) are used to visualize the robot's position and path.

[1292] This allows residents to check the robot's status in real time and move around with peace of mind.

[1293] Example

[1294] For example, when a resident moves from their room to the entrance of an apartment building, they follow the steps below.

[1295] 1. Request transmission from user terminal

[1296] The user starts the application on their device and inputs a pickup request, which includes destination information and current location.

[1297] 2. Request Processing on the Server

[1298] The server receives the request, analyzes it, and sends instructions to the most suitable robot.

[1299] 3. Robot movement and resident guidance

[1300] The robot receives instructions, starts moving autonomously, connects to the electric senior cart, and detects and avoids obstacles as necessary.

[1301] The robot will notify the user's device or smart glasses of its arrival time and provide guidance along the way.

[1302] 4. Displaying real-time information

[1303] The robot's position and movement path are visualized through smart glasses or a head-mounted display.

[1304] Prompt example

[1305] Possible input prompts for a generative AI model include:

[1306] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[1307] This system allows residents to travel from their homes to their destinations efficiently and safely with peace of mind. This completes the embodiment of the present invention.

[1308] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1309] Step 1:

[1310] The application is started on the user terminal and a transportation request is input.

[1311] The input includes current location and destination information.

[1312] The user terminal transmits the input data to the server.

[1313] Input: Current location and destination information from the user

[1314] Output: Sending request data to the server

[1315] Step 2:

[1316] The server receives the request sent from the user terminal.

[1317] The server analyzes the request data and obtains the user's current location and destination information.

[1318] Input: Request data sent from the user terminal

[1319] Output: Parsed current location and destination information

[1320] Step 3:

[1321] The server checks the current location of the robots to select the most suitable robot.

[1322] A movement instruction is generated for the optimal robot and sent to the robot.

[1323] Input: Current location and destination information, and robot location data

[1324] Output: Movement instructions to the robot

[1325] Step 4:

[1326] The robot receives movement instructions from the server and begins moving autonomously.

[1327] The robot connects to an electric senior car and moves towards its destination.

[1328] Input: Move instruction from server

[1329] Output: Robot starts moving

[1330] Step 5:

[1331] Sensors installed on the robot detect obstacles and select a safe route for driving.

[1332] It processes sensor data in real time and performs data calculations to avoid obstacles.

[1333] Input: Sensor data from the surrounding environment

[1334] Output: Path information for obstacle avoidance

[1335] Step 6:

[1336] The robot sends its location information and estimated arrival time to the server.

[1337] The server provides this information to user devices, smart glasses, and head-mounted displays in real time.

[1338] Input: Robot location and estimated arrival time

[1339] Output: Real-time notifications to user devices and smart glasses

[1340] Step 7:

[1341] The user visually checks the robot's location and movement route using smart glasses or a head-mounted display.

[1342] This allows users to travel with peace of mind.

[1343] Input: Location information and movement route data from the server

[1344] Output: Real-time display as visual information

[1345] Step 8:

[1346] When the robot arrives at its destination, it sends an arrival notification to the server.

[1347] The server notifies the user terminal of the arrival information, and the series of movements is completed.

[1348] Input: Arrival notification from the robot

[1349] Output: Arrival information notification to user terminal

[1350] Prompt example

[1351] Possible input prompts for a generative AI model include:

[1352] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[1353] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1354] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions, and is realized by linking the user terminal, server, robot, electric senior car, and emotion engine.

[1355] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[1356] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereafter referred to as the robot) based on the user's location and destination information. The server monitors the robot's location and status in real time, and sends control signals as necessary to remotely control the robot's operation.

[1357] The robot receives instructions from the server and heads to the designated entrance lobby to connect the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. It is also capable of smooth communication with residents using AI technology. It is also equipped with an emotion engine that can analyze the user's emotional state and respond appropriately. The emotion engine recognizes emotions based on the user's facial expressions, voice, and behavioral data, and the robot responds appropriately.

[1358] The specific operation of the system will be described below.

[1359] Consider a scenario where Resident A goes out and wants to return to his room at home.

[1360] 1. User (A):

[1361] Open the smartphone app and enter your pickup request.

[1362] 2. Device (app):

[1363] A request for transportation from Mr. A is sent to the server.

[1364] 3. Server:

[1365] Receive the request and confirm Mr. A's location and destination.

[1366] It instructs the nearest robot to move and perform a series of actions.

[1367] 4. Robots:

[1368] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[1369] The system uses an emotion engine to analyze Mr. A's emotional state and greets him appropriately upon arrival.

[1370] 5. User (A):

[1371] Check the senior car connected to the robot in the entrance lobby and get in.

[1372] Travel with peace of mind thanks to the robot's voice guidance.

[1373] 6. Robots:

[1374] It uses sensors to avoid obstacles and safely drive towards your room in the home.

[1375] If Mr. A seems anxious while driving, the emotion engine will detect this and reassure him with appropriate words.

[1376] 7. Terminal:

[1377] The robot's location is displayed in real time and provided to Mr. A.

[1378] 8. Robots:

[1379] When they arrive at their destination, they inform Mr. A of their arrival and greet him in a friendly manner.

[1380] 9. User (A):

[1381] I get off the senior car and enter my room at home.

[1382] 10. Robots:

[1383] The server is notified that the transfer is complete, and the emotional state of Mr. A recognized by the emotion engine is also sent.

[1384] If there is no next task, move to an appropriate waiting position.

[1385] This series of processes will realize a system that allows users to move around the apartment building safely and efficiently while taking their emotions into consideration. The emotion engine's response is expected to provide an elderly-friendly environment and improve the quality of life.

[1386] The processing flow will be explained below.

[1387] Step 1:

[1388] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[1389] Step 2:

[1390] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[1391] Step 3:

[1392] The device transmits the location information along with the pickup request entered by the user to the server, including the user's destination information.

[1393] Step 4:

[1394] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[1395] Step 5:

[1396] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[1397] Step 6:

[1398] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[1399] Step 7:

[1400] When the robot arrives at the entrance lobby, it notifies the user of its arrival with a voice prompt. It also uses a built-in emotion engine to analyze the user's emotions and provides voice prompts and greetings according to the user's emotional state.

[1401] Step 8:

[1402] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[1403] Step 9:

[1404] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[1405] Step 10:

[1406] The terminal transmits the user's "depart" instruction to the server.

[1407] Step 11:

[1408] The server sends a "go" command to the robot.

[1409] Step 12:

[1410] The robot receives the "depart" command and begins autonomously moving toward the room in the home. It uses built-in sensors to detect obstacles and drive safely. It also uses an emotion engine to monitor the user's emotions.

[1411] Step 13:

[1412] The robot analyzes the user's emotions while moving, and if it detects anxiety or tension, it will provide voice guidance and dialogue to provide comfort.

[1413] Step 14:

[1414] The terminal displays the robot's location information in real time and provides it to the user.

[1415] Step 15:

[1416] When the robot arrives at its destination, it notifies the user and speaks in a friendly manner. Based on the analysis results of its emotion engine, it checks whether the user is calm.

[1417] Step 16:

[1418] The user gets out of the mobility scooter and enters a room in their home.

[1419] Step 17:

[1420] The robot notifies the server that the transfer is complete and also sends the user's emotional state as recognized by the emotion engine.

[1421] Step 18:

[1422] The server marks the robot's task as completed and checks to see if there are any other tasks waiting.

[1423] Step 19:

[1424] If there is no next task, the robot moves to an appropriate waiting position and waits for the next instruction.

[1425] This series of processes will realize a system that utilizes an emotion engine to allow elderly people to move safely around an apartment building while also taking into consideration their emotional state.

[1426] Example 2

[1427] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1428] Conventional outing support systems simply support movement without considering the user's emotional state. As a result, elderly and emotionally unstable users often feel anxious while out and about. Another issue is that communication with the user is not smooth, making it difficult for the user to feel familiar with the system.

[1429] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a user terminal operated by the resident to provide outing assistance, means for receiving and managing requests sent from the user terminal, means including a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, an electric senior car for transporting the resident while connected to the robot, means for detecting obstacles using sensors mounted on the robot and driving safely, means for remotely controlling the operation of the robot using control signals sent from the server, means for analyzing the emotional state of the resident using an emotion engine, and means for providing appropriate greetings or calls based on the user's emotions. This enables safe and friendly outing assistance that takes the user's emotional state into consideration.

[1430] A "user terminal" is an electronic device operated by a user, and includes a smartphone, a tablet, and the like.

[1431] A "server" is a computer system that receives and manages requests sent from user terminals.

[1432] A "request" is request information sent from a user terminal to a server, and includes instructions for transportation or movement.

[1433] The "robot" is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car.

[1434] An "electric senior car" is an electric mobility device that is connected to a robot to safely transport residents.

[1435] "Sensors" are devices installed on robots to detect obstacles, and include LIDAR and cameras.

[1436] A "control signal" is an instruction sent from the server to the robot to remotely control the robot's operations.

[1437] An "emotion engine" is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response.

[1438] "Appropriate greetings and calls" are communication actions that the robot takes toward the user based on the analysis results of the emotion engine.

[1439] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions. The main components of the system are a user terminal, a server, a robot, an electric senior car, and an emotion engine, all of which work in conjunction with each other.

[1440] User terminal

[1441] A user terminal is an electronic device operated by a user, and includes smartphones, tablets, etc. A dedicated application is installed on the device, and the user uses this application to input a pickup request. For example, simply inputting a request such as "I want to go to my room at home" initiates a series of operations in the system.

[1442] server

[1443] The server is a computer system that receives and manages requests sent from the user's terminal. The server analyzes the request data and sends appropriate instructions to the robot based on the user's current location and destination. During this process, the server constantly monitors the robot's location and status in real time, and sends control signals each time to remotely control the robot's operation.

[1444] robot

[1445] The robot is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car. The robot is equipped with high-precision sensors and can move safely while detecting obstacles using LIDAR and cameras. The robot also has an emotion engine that recognizes emotions based on the user's facial expressions, voice, and behavioral data. For example, if the user appears anxious, the robot can reassure them with appropriate words such as, "Don't worry. We'll soon arrive at our destination."

[1446] Electric senior car

[1447] An electric senior car is an electric mobility device that connects to a robot to safely transport residents. Because it is controlled by the robot, the user does not need to drive it themselves. It allows residents to travel safely and smoothly to specific destinations within the apartment complex.

[1448] Emotion Engine

[1449] The emotion engine is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response. The emotion engine uses data obtained from a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing their emotional state in real time. This allows the robot to respond appropriately to the user, providing a sense of security, especially for elderly and emotionally unstable users.

[1450] Add specific examples to the description

[1451] For example, if resident A wants to return to their room after being out, the following scenario can be considered: First, resident A opens the smartphone app and enters a request for transportation. The app sends the request to the server, which receives and analyzes the request, and then sends instructions to the nearest robot. Based on instructions from the server, the robot heads to the entrance lobby and connects its electric senior car. The robot then uses its emotion engine to analyze resident A's emotional state and responds with an appropriate greeting, such as, "Hello, resident A. How are you feeling today?" resident A gets into the senior car and follows the robot's guidance, feeling reassured as they travel. Finally, when the robot arrives at its destination and resident A gets out of the senior car, the robot notifies the server that transportation is complete and waits for the next task.

[1452] Prompt Sentence Examples

[1453] Below are some example prompts for a generative AI model:

[1454] "Please explain how an autonomous mobile robot equipped with an emotion engine can analyze the emotional state of its user."

[1455] "Please explain in detail the procedure for welcoming a robot using the emotion engine."

[1456] This not only allows users to use the system with peace of mind, but also provides an environment that is particularly friendly to elderly people.The present invention aims to improve the quality of life of users.

[1457] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1458] Step 1:

[1459] The user opens a dedicated app installed on their smartphone or tablet and enters a transportation request.

[1460] Specific operation: The user launches the app, enters their destination (e.g., their room number at home), and presses the pick-up button.

[1461] Input: Trip request information entered by the user (destination, current location).

[1462] Output: Trip request data generated within the app.

[1463] Step 2:

[1464] The terminal sends the user's transportation request to the server.

[1465] Specific behavior: The app sends a request containing the user's location information (GPS data) and destination information to the server as an HTTP POST request.

[1466] Input: The generated trip request data.

[1467] Output: The request data sent to the server.

[1468] Step 3:

[1469] The server receives the request and verifies the user's location and destination.

[1470] Specific operation: The server analyzes the request data and identifies the user's current location (for example, the entrance to an apartment building).

[1471] Input: Request data sent from the device.

[1472] Output: Parsed user location and destination information.

[1473] Step 4:

[1474] The server instructs the nearest robot to move and perform a series of actions.

[1475] Specific operation: The server checks the robot's current location, calculates the optimal route, and then sends instructions such as "head to the entrance, connect the senior cart, and welcome the user."

[1476] Input: User's current location, destination, and robot's current location data.

[1477] Output: Movement and motion instruction data sent to the robot.

[1478] Step 5:

[1479] The robot receives instructions from the server, heads to the entrance lobby, and connects the electric senior car.

[1480] Specific operation: The robot uses its own navigation system to autonomously navigate to the entrance lobby and physically connect with the senior car.

[1481] Input: Movement and action instruction data sent from the server.

[1482] Output: Arrival at the entrance lobby and the connection status of the mobility scooter.

[1483] Step 6:

[1484] The robot uses an emotion engine to analyze the user's emotional state and greets them appropriately upon arrival.

[1485] Specific operation: The robot uses a camera and microphone to analyze the user's facial expressions and tone of voice, and then greets them with a greeting such as, "Hello, Mr. A. How are you feeling today?"

[1486] Input: User's facial expression data, voice data.

[1487] Output: Analysis results and a voice message based on them.

[1488] Step 7:

[1489] The user checks the senior car connected to the robot in the entrance lobby and gets in.

[1490] Specific operation: The user follows the robot's instructions to get into the senior car.

[1491] Input: User confirmation and ride operation.

[1492] Output: Riding in a mobility scooter.

[1493] Step 8:

[1494] The robot's voice guidance provides users with a sense of security as they move around.

[1495] What it does: The robot provides voice guidance such as, "Don't worry, we're on our way to your next destination."

[1496] Input: Robot voice guidance data.

[1497] Output: A voice message to the user.

[1498] Step 9:

[1499] The robot uses sensors to avoid obstacles and safely navigates towards your room.

[1500] How it works: The robot uses a LIDAR sensor to scan the surrounding environment and automatically selects a safe route to move.

[1501] Input: Sensor data, environmental data.

[1502] Output: Obstacle-avoiding route information and safe movement status.

[1503] Step 10:

[1504] If the user appears anxious while the robot is driving, the emotion engine will detect this and provide appropriate reassurance.

[1505] What it does: The emotion engine detects the user's anxiety and provides reassuring words such as, "You're almost there."

[1506] Input: User's facial expression data, voice data.

[1507] Output: Analysis results and a voice message based on them.

[1508] Step 11:

[1509] The robot arrives at the destination and notifies the user of its arrival while speaking in a friendly manner.

[1510] Specific operation: The robot will provide guidance such as, "Mr. A, you have arrived at your destination. Thank you for your hard work."

[1511] Input: Destination arrival data.

[1512] Output: A voice message to the user.

[1513] Step 12:

[1514] The user gets out of the mobility scooter and enters a room in their home.

[1515] Specific actions: The user gets out of the mobility scooter, unlocks the door to their home, and enters.

[1516] Input: User exit and entry operations.

[1517] Output: The user's room entry status.

[1518] Step 13:

[1519] The robot notifies the server that the transfer is complete and also transmits the user's emotional state as recognized by the emotion engine.

[1520] Specific operation: The robot uploads the data of the completed transfer to the server and also sends the emotion data.

[1521] Input: Pick-up and drop-off completion data, emotional state data.

[1522] Output: Data sent to the server.

[1523] Step 14:

[1524] If the robot has no next task, it moves to an appropriate waiting position.

[1525] Specific behavior: The robot moves to the waiting station and waits for further instructions.

[1526] Input: Next task availability data.

[1527] Output: Waiting in standby position.

[1528] (Application example 2)

[1529] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1530] There is a need for systems that allow elderly people and people who require physical assistance to shop safely and with peace of mind in physical stores. It is also important to provide appropriate support according to their emotional state. Existing systems have had problems with insufficient emotion recognition and are unable to adequately respond to the anxiety and confusion of elderly people.

[1531] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and managing requests sent from a user terminal, means for controlling an autonomous mobile device that moves autonomously based on instructions sent from the server and connects to an electric vehicle, and means for recognizing the emotional state of residents using an emotion engine installed in the autonomous mobile device and taking appropriate action. This enables elderly people and people who require physical assistance to move safely within a store and shop with peace of mind. Furthermore, the emotion engine provides appropriate support according to the user's emotional state, thereby improving their sense of security and satisfaction.

[1532] "Residents" are users of the system, primarily elderly people and those who require physical assistance.

[1533] "User terminal" refers to electronic devices such as smartphones and tablets that are operated by residents.

[1534] A "request" refers to information sent from a user terminal to a server to request support or movement.

[1535] The term "server" refers to a computer system that manages requests sent from user terminals and sends instructions to autonomous mobile devices.

[1536] "Autonomous mobile equipment" refers to robots and similar equipment that move autonomously and connect to electric vehicles based on instructions from a server to support residents.

[1537] "Electric vehicle" refers to an electric senior car or other electrically powered vehicle that is connected to an autonomous mobility device and used to transport residents.

[1538] "Sensor" refers to a device installed on an autonomous mobile device to detect obstacles in the surrounding area.

[1539] "Emotion engine" refers to artificial intelligence technology that recognizes residents' emotional state and responds appropriately.

[1540] "Artificial intelligence technology" refers to technology used by autonomous mobile devices to facilitate communication with residents.

[1541] This invention is a system that helps elderly people and people who require physical assistance to shop safely in physical stores. The system is composed of components including a server, user terminals, autonomous mobile devices, electric vehicles, sensors, an emotion engine, and artificial intelligence technology.

[1542] First, the user device is a smartphone or tablet. Residents submit support requests through a dedicated application. This request is sent from the device to a server. The server receives the request and analyzes the resident's location and the type of support required. Based on the analysis results, it sends instructions to the nearest autonomous mobile device.

[1543] The autonomous mobile device receives instructions from the server and moves to a designated location within the store. This device connects to an electric vehicle (such as an electric cart or senior cart) to transport the resident. The autonomous mobile device is equipped with sensors that detect and avoid obstacles in the surrounding area. It is also equipped with an emotion engine that analyzes the resident's facial expressions, voice, and behavior to recognize their emotional state and respond appropriately.

[1544] The server monitors the location of the autonomous mobile device in real time and remotely controls it as needed, using a high-precision control system.

[1545] As a specific example, consider the following scenario.

[1546] Scenario: Elderly person B goes shopping at a supermarket

[1547] 1. User (Mr. B):

[1548] Open the smartphone app, select "Shopping Support," and submit a support request.

[1549] 2. Device (app):

[1550] Send Mr. B's shopping support request to the server.

[1551] 3. Server:

[1552] Receive the request and confirm Mr. B's location and the support he needs.

[1553] Send instructions to the nearest autonomous mobile device.

[1554] 4. Autonomous mobile devices:

[1555] Following the server's instructions, I headed towards the entrance of the supermarket.

[1556] The emotional engine is used to analyze Mr. B's emotional state and provide an appropriate greeting or greeting upon arrival.

[1557] 5. Person B:

[1558] The user contacts an autonomous mobile device near the entrance and requests assistance with shopping.

[1559] Follow the voice guidance of the autonomous mobile device and start shopping with peace of mind.

[1560] 6. Autonomous mobile devices:

[1561] As they move around the store together, they guide B to products based on his shopping list.

[1562] Move safely while detecting obstacles.

[1563] If Person B feels anxious, the emotion engine will detect this and provide appropriate words to reassure them.

[1564] Hardware and Software Used

[1565] User device: Smartphone (iOS / Android)

[1566] Server: Cloud server (AWS or Google Cloud) + Flask (Python web server framework)

[1567] Autonomous mobile device: Autonomous mobile robot (equipped with LiDAR sensor, camera, and microphone)

[1568] Emotion Engine: Emotion Recognition Model with TensorFlow / Keras

[1569] An example prompt is:

[1570] "Imagine that Person B goes to the supermarket and asks an autonomous mobile device to help him with his shopping. Person B is feeling a little anxious. How would the autonomous mobile device respond to reassure Person B?"

[1571] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1572] Step 1:

[1573] User (Mr. B)

[1574] Open the smartphone app, select "Shopping Support," and submit a support request.

[1575] Input: B's location information, request details

[1576] Output: Support request data

[1577] Step 2:

[1578] Device (app)

[1579] Send Mr. B's shopping support request to the server.

[1580] Input: Support request data

[1581] Output: Request data sent to the server

[1582] Step 3:

[1583] server

[1584] The system receives the request, checks B's location and the support required, analyzes the request data, and sends instructions to the nearest autonomous mobile device.

[1585] Input: Request data sent to the server

[1586] Data processing / calculation: Analyzes the request and selects the most suitable autonomous mobile device

[1587] Output: Instruction data sent to the autonomous mobile device

[1588] Step 4:

[1589] Autonomous mobile equipment

[1590] It receives instructions from the server and moves to a designated location within the store. During its movement, it uses sensors to detect surrounding obstacles and proceed safely.

[1591] Input: Instruction data sent to the autonomous mobile device

[1592] Data processing / calculation: Calculating movement routes based on location information, detecting obstacles using sensors

[1593] Output: Action instructions to move safely

[1594] Step 5:

[1595] Autonomous mobile equipment

[1596] Upon arrival, the emotion engine is used to analyze Mr. B's emotional state and an appropriate greeting or greeting is given.

[1597] Input: Person B's facial expression and voice data

[1598] Data processing / calculation: Emotion recognition using TensorFlow / Keras, and decision on how to respond based on the analysis results

[1599] Output: An appropriate response to Person B

[1600] Step 6:

[1601] User (Mr. B)

[1602] Users contact an autonomous mobile device near the entrance and request shopping assistance. They then follow the audio guidance provided by the autonomous mobile device and begin shopping with peace of mind.

[1603] Input: Autonomous mobile device information

[1604] Output: Person B's actions

[1605] Step 7:

[1606] Autonomous mobile equipment

[1607] As the robot moves around the store together with B, it guides him to products based on his shopping list. Along the way, it again uses sensors to detect and avoid obstacles. If B feels confused or anxious, the emotion engine detects this and uses appropriate words to reassure him.

[1608] Input: B's shopping list, facial expressions, voice data, sensor information

[1609] Data processing / calculation: Navigation to product locations based on shopping lists, emotion recognition, obstacle avoidance

[1610] Output: Product guidance, obstacle avoidance behavior, appropriate response

[1611] Step 8:

[1612] Autonomous mobile equipment

[1613] Once the customer has finished shopping, they will be guided to the cash register and supported until payment is complete.

[1614] Input: B's actions, cash register location information

[1615] Data processing / calculation: Secure route calculation, payment completion confirmation

[1616] Output: Instructions to cashier, necessary responses

[1617] Step 9:

[1618] server

[1619] It notifies the robot that the shopping support is complete and records the emotional state of the robot according to the emotion engine. If there is no next task, the robot moves to an appropriate waiting position.

[1620] Input: Completion notification from autonomous mobile device, emotional state data

[1621] Data processing / calculation: Confirmation of task completion, movement plan to waiting position

[1622] Output: Updates to system logs, instructions to move autonomous mobile devices to standby positions

[1623] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1624] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1625] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1626] [Fourth embodiment]

[1627] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1628] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1629] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1630] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1631] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1632] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1633] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1634] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1635] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1636] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1637] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1638] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1639] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1640] This invention is a system that supports residents in moving from their homes safely and efficiently, and is realized by linking user terminals, servers, robots, and electric senior cars.

[1641] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[1642] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereinafter referred to as the robot). The server manages the user's location information and request content, and monitors the robot's location and status in real time.

[1643] The robot receives instructions from the server and heads to the designated entrance lobby, where it connects the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. AI technology is also used to enable smooth communication with residents. For example, the robot can notify the user of its arrival time in advance and provide directions along the way. The robot connects the senior car and heads to the user's entrance lobby.

[1644] The user gets into the senior car in the entrance lobby and travels with the robot to their room in their home. During this time, the robot uses sensors and AI technology to drive safely, and can also accept remote control via control signals sent from the server as needed. The user's device can check the robot's location in real time, allowing the user to travel with peace of mind.

[1645] As a concrete example, we will show a scenario in which an elderly resident, Mr. A, wants to return to his room at home after going out.

[1646] 1. User (A):

[1647] Open the smartphone app and enter your pickup request.

[1648] 2. Device (app):

[1649] Send A's request to the server.

[1650] 3. Server:

[1651] Upon receiving the request, confirm Mr. A's location and his room number at home.

[1652] It instructs the nearest robot to move to the entrance lobby and connect the senior cart.

[1653] 4. Robots:

[1654] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[1655] 5. Robot and Person A:

[1656] We welcome Mr. A in the entrance lobby and guide him to the senior car.

[1657] After checking for safety, I head to my room.

[1658] 6. User Device:

[1659] The robot's location is displayed in real time, providing Mr. A with a sense of security.

[1660] 7. Arrival:

[1661] We arrive in front of Mr. A's house and park the mobility scooter.

[1662] The robot notifies the server that the transfer is complete.

[1663] This system provides an environment where elderly people can move around the apartment building safely and efficiently, and is expected to promote social participation and maintain health. This invention is a practical means to improve the quality of life for elderly people and also brings benefits to apartment building managers.

[1664] The processing flow will be explained below.

[1665] Step 1:

[1666] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[1667] Step 2:

[1668] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[1669] Step 3:

[1670] The terminal transmits the pickup request input by the user to the server, including the user's location information and destination information.

[1671] Step 4:

[1672] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[1673] Step 5:

[1674] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[1675] Step 6:

[1676] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[1677] Step 7:

[1678] When the robot arrives at the entrance lobby, it will notify the user of its arrival via voice guidance.

[1679] Step 8:

[1680] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[1681] Step 9:

[1682] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[1683] Step 10:

[1684] The terminal transmits the user's "depart" instruction to the server.

[1685] Step 11:

[1686] The server sends a "go" command to the robot.

[1687] Step 12:

[1688] The robot receives the "depart" command and begins autonomously moving towards the room in the home. It uses its built-in sensors to detect obstacles and drive safely.

[1689] Step 13:

[1690] The terminal obtains the robot's location information in real time and provides it to the user.

[1691] Step 14:

[1692] When the robot arrives at its destination, it will notify the user by voice.

[1693] Step 15:

[1694] The user gets out of the mobility scooter and enters a room in their home.

[1695] Step 16:

[1696] The robot notifies the server that the transfer is complete.

[1697] Step 17:

[1698] The server marks the robot's task as completed and checks for the robot's next task.

[1699] Step 18:

[1700] The robot moves to an appropriate position to wait for the next instruction if there is no next task.

[1701] This series of processes will result in a system that will enable safe and secure transportation for the elderly.

[1702] Example 1

[1703] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1704] In modern society, there is a demand for support systems that enable elderly people and people with disabilities to travel efficiently and safely. However, current systems lack the technical capabilities to provide real-time location information, detect obstacles for safe driving, and provide autonomous mobility support. This makes it difficult to provide an environment where elderly people and people with disabilities can go out with peace of mind.

[1705] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1706] In this invention, the server includes a data analysis means for analyzing requests from the information processing device and issuing appropriate instructions, a mechanism means for the mechanical device to move in response to instructions and connect to the mobile device at a specified location, and a mechanical device that moves autonomously and connects to the mobile device based on instructions sent from the data processing device, thereby enabling elderly people and people with disabilities to move efficiently and safely.

[1707] "User" refers to a person who uses the system to receive mobility assistance.

[1708] The term "information processing device" refers to a terminal device operated by a user, which inputs and transmits requests.

[1709] "Data processing device" refers to a central system that receives and analyzes requests sent from an information processing device and issues instructions.

[1710] The term "mechanical device" refers to a device such as a robot that operates autonomously based on instructions from a data processing device and that supports the movement of a user by connecting to a mobile device.

[1711] "Mobility device" refers to a device such as an electric senior car or wheelchair that is connected to a mechanical device to transport a user.

[1712] "Sensor" refers to a device installed on a machine to detect obstacles and sense the surrounding environment.

[1713] "Data analysis means" refers to a method or technology for analyzing a request sent from an information processing device and creating appropriate instructions.

[1714] The term "mechanical means" refers to a mechanism by which a mechanical device connects a moving device based on an instruction.

[1715] "Artificial intelligence technology" refers to machine learning and natural language processing technologies used by machines to smoothly communicate with users.

[1716] This invention is a system that supports the elderly and physically disabled to move around safely and efficiently. The system consists of a user terminal, a data processing device (server), a mechanical device (robot), and a mobility device (electric senior car). The specific hardware and software configurations, as well as data processing and calculation methods, are explained below.

[1717] Hardware and software used

[1718] User terminal: Includes information processing devices such as smartphones and tablets, on which dedicated applications are installed.

[1719] Data Processing Unit (Server): The central system that analyzes requests and issues appropriate instructions to the robot. It includes a database and a communication interface.

[1720] Mechanical device (robot): A robot capable of autonomous movement, equipped with high-precision sensors and artificial intelligence (AI) technology.

[1721] Mobility device (electric senior car): A means of transporting the user in cooperation with the robot. It operates by being connected to the robot.

[1722] Program processing overview

[1723] User device operation

[1724] Users input and send their transportation requests using a dedicated application, which includes functions such as inputting requests, obtaining current locations, and displaying real-time location information.

[1725] Server Processing

[1726] The server analyzes requests received from the user's device and confirms the user's current location and destination. Based on the analysis results, it issues instructions to the nearest robot, including moving to the entrance lobby and connecting to an electric senior car. The server also monitors the robot's location and status in real time and sends control signals as necessary.

[1727] Robot movement

[1728] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves and connects to the electric senior car. Upon arrival, the robot greets the user and guides them to the senior car. During the journey, it makes full use of AI technology to check for safety and communicates smoothly with the user as needed.

[1729] Specific example explanation

[1730] When elderly resident A goes out and wants to return to his room, the following steps are taken:

[1731] 1. User: Opens the smartphone app and enters a pickup request.

[1732] Example prompt: How do I enter a destination using my smartphone?

[1733] 2. Terminal: The user terminal sends a request to the server.

[1734] Sample prompt: Describe the process a user goes through to submit a trip request in your app.

[1735] 3. Server: Receives the request, analyzes it, and sends instructions to the nearest robot.

[1736] Sample prompt: Please explain in detail how the server parses the request.

[1737] 4. Robot: Following instructions, it heads to the entrance lobby and connects the electric senior car.

[1738] Sample prompt: How can the robot navigate to the designated location and connect the mobility scooter?

[1739] 5. Robot: Welcomes the user in the entrance lobby and guides them to the mobility scooter.

[1740] Sample prompt: Please explain in detail the steps the robot takes to pick up the user.

[1741] This system provides an environment where elderly people and people with disabilities can move around safely and efficiently.

[1742] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1743] Step 1:

[1744] Enter your transportation request

[1745] User:

[1746] The resident opens the smartphone app, taps the "Request a Pickup" button on the app's home screen, and selects their room as the destination.

[1747] Input: Destination selection, user location information

[1748] Output: Pickup request information (user ID, current location, destination, timestamp)

[1749] Step 2:

[1750] Submit a Pickup Request

[1751] Device:

[1752] The user device sends A's request information to the server. When the user taps the send button, the request information is sent to the server via the network.

[1753] Input: Pickup request information

[1754] Output: HTTP request to the server

[1755] Step 3:

[1756] Receiving and parsing the request

[1757] server:

[1758] The server receives and analyzes the request sent from the user's device. The server stores the received request information (user ID, room number, current location) in a database and assigns the nearest robot.

[1759] Input: HTTP request (transport request information)

[1760] Output: Instructions to the robot (destination, current location, user information)

[1761] Step 4:

[1762] Robot instructions

[1763] server:

[1764] The server instructs the nearest robot to move to the entrance lobby and connect to the electric senior car. The server sends an HTTP request to the robot, instructing it to move and connect to the senior car. After the robot receives the command, it begins to act.

[1765] Input: Instructions for the robot

[1766] Output: Robot starts moving

[1767] Step 5:

[1768] Robot movement and connection

[1769] robot:

[1770] The robot receives instructions from the server and heads to the designated entrance lobby. It uses high-precision sensors to avoid obstacles as it moves, and the mechanism for connecting the electric senior car to it is activated.

[1771] Input: Instructions from the server

[1772] Output: Arrival at the entrance lobby and docking of the mobility scooter

[1773] Step 6:

[1774] Meet and greet users

[1775] robot:

[1776] The robot greets Mr. A in the entrance lobby and guides him to the senior car. It calls out to Mr. A by voice and guides him to the boarding position of the senior car. The robot uses AI-based natural language processing to converse with him.

[1777] Input: User pickup instructions

[1778] Output: Directions to the user's mobility scooter

[1779] Step 7:

[1780] Providing real-time information

[1781] Device:

[1782] The user's device displays the robot's real-time location information. The app displays a map on the screen, and the current locations of the robot and the user are updated in real time, giving the user a sense of security.

[1783] Input: Robot location

[1784] Output: Real-time map display

[1785] Step 8:

[1786] Completing the move

[1787] robot:

[1788] The robot and the senior car deliver Mr. A to his room at home. After arriving, the robot notifies the server that the transfer is complete. The server receives this notification and records the completion of the transfer in the database.

[1789] Input: Arrival at destination

[1790] Output: Notification of completion of transfer to server

[1791] (Application example 1)

[1792] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1793] Existing technologies for supporting elderly and disabled residents to travel safely and efficiently from their homes to their destinations face challenges in terms of safety and convenience. Furthermore, there is a lack of ways for residents to visually check real-time information while traveling, making it difficult for them to travel safely. Furthermore, technology is needed to enable smooth communication between residents and robots.

[1794] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1795] In this invention, the server includes a user terminal operated by the resident to provide assistance with going out, a means for receiving and managing requests sent from the user terminal, a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, the electric senior car for transporting the resident while connected to the robot, a means for detecting obstacles using sensors mounted on the robot and safely driving it, a means for remotely controlling the robot's operation using control signals sent from the server, and a means for visualizing the robot's position information and movement route in real time using smart glasses or a head-mounted display. This enables residents to travel safely and efficiently from their homes to their destinations with peace of mind.

[1796] A "user terminal" is an electronic device operated by residents to assist them in going out, and includes smartphones, tablets, and the like.

[1797] A "server" is a central management device that receives and manages requests sent from user terminals.

[1798] A "robot" is a mechanical device that moves autonomously based on instructions sent from a server and connects to an electric senior car.

[1799] An "electric senior car" is an electric vehicle that is connected to a robot and used to transport residents.

[1800] A "sensor" is a device installed on a robot that detects obstacles.

[1801] A "control signal" is an operation command signal sent from the server to the robot.

[1802] "Remote control means" refers to a method of remotely controlling the robot's movements using control signals sent from a server.

[1803] "Smart glasses" are wearable devices that can visually display the robot's position information and movement route.

[1804] A "head-mounted display" is a device worn by a user on the head that displays information, and is used to visualize the position information and movement path of a robot.

[1805] "Real-time information" refers to data or information that provides an immediate picture of the current situation.

[1806] "AI technology" refers to artificial intelligence technology that is installed in robots and enables smooth communication with residents.

[1807] "Visualization means" refers to a method for visualizing the robot's position information and movement path using smart glasses, a head-mounted display, or the like.

[1808] This invention is a support system for elderly people and people with disabilities to move from their homes to their destinations safely and efficiently. A specific embodiment of the system is described below.

[1809] System configuration

[1810] The system includes a user terminal, a server, a robot, an electric senior car, smart glasses, and a head-mounted display.

[1811] 1. User Device

[1812] A user terminal is an electronic device, such as a smartphone or tablet, that a resident uses to input a transportation request.

[1813] The application installed on the user terminal has a function for sending requests and a function for displaying real-time location information.

[1814] 2. Server

[1815] The server receives and analyzes the request sent from the user terminal.

[1816] The server manages the user's location information and request details, and sends instructions to the most appropriate robot.

[1817] The server is built using cloud services such as AWS, and communication is carried out in real time using WebSocket.

[1818] 3. Robot

[1819] The autonomous mobile robot moves automatically and connects to the electric senior car based on instructions from the server.

[1820] The robot is equipped with high-precision sensors (e.g., LIDAR, cameras) to detect and avoid obstacles.

[1821] Using AI technology (e.g. TensorFlow), we communicate smoothly with residents and provide guidance along the way.

[1822] 4. Electric senior car

[1823] The electric senior car is an electric vehicle connected to a robot to transport residents.

[1824] Senior cars provide safe and comfortable transportation.

[1825] 5. Smart Glasses and Head-Mounted Displays

[1826] Smart glasses or head-mounted displays (e.g., Google Glass, Microsoft HoloLens) are used to visualize the robot's position and path.

[1827] This allows residents to check the robot's status in real time and move around with peace of mind.

[1828] Example

[1829] For example, when a resident moves from their room to the entrance of an apartment building, they follow the steps below.

[1830] 1. Request transmission from user terminal

[1831] The user starts the application on their device and inputs a pickup request, which includes destination information and current location.

[1832] 2. Request Processing on the Server

[1833] The server receives the request, analyzes it, and sends instructions to the most suitable robot.

[1834] 3. Robot movement and resident guidance

[1835] The robot receives instructions, starts moving autonomously, connects to the electric senior cart, and detects and avoids obstacles as necessary.

[1836] The robot will notify the user's device or smart glasses of its arrival time and provide guidance along the way.

[1837] 4. Displaying real-time information

[1838] The robot's position and movement path are visualized through smart glasses or a head-mounted display.

[1839] Prompt example

[1840] Possible input prompts for a generative AI model include:

[1841] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[1842] This system allows residents to travel from their homes to their destinations efficiently and safely with peace of mind. This completes the embodiment of the present invention.

[1843] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1844] Step 1:

[1845] The application is started on the user terminal and a transportation request is input.

[1846] The input includes current location and destination information.

[1847] The user terminal transmits the input data to the server.

[1848] Input: Current location and destination information from the user

[1849] Output: Sending request data to the server

[1850] Step 2:

[1851] The server receives the request sent from the user terminal.

[1852] The server analyzes the request data and obtains the user's current location and destination information.

[1853] Input: Request data sent from the user terminal

[1854] Output: Parsed current location and destination information

[1855] Step 3:

[1856] The server checks the current location of the robots to select the most suitable robot.

[1857] A movement instruction is generated for the optimal robot and sent to the robot.

[1858] Input: Current location and destination information, and robot location data

[1859] Output: Movement instructions to the robot

[1860] Step 4:

[1861] The robot receives movement instructions from the server and begins moving autonomously.

[1862] The robot connects to an electric senior car and moves towards its destination.

[1863] Input: Move instruction from server

[1864] Output: Robot starts moving

[1865] Step 5:

[1866] Sensors installed on the robot detect obstacles and select a safe route for driving.

[1867] It processes sensor data in real time and performs data calculations to avoid obstacles.

[1868] Input: Sensor data from the surrounding environment

[1869] Output: Path information for obstacle avoidance

[1870] Step 6:

[1871] The robot sends its location information and estimated arrival time to the server.

[1872] The server provides this information to user devices, smart glasses, and head-mounted displays in real time.

[1873] Input: Robot location and estimated arrival time

[1874] Output: Real-time notifications to user devices and smart glasses

[1875] Step 7:

[1876] The user visually checks the robot's location and movement route using smart glasses or a head-mounted display.

[1877] This allows users to travel with peace of mind.

[1878] Input: Location information and movement route data from the server

[1879] Output: Real-time display as visual information

[1880] Step 8:

[1881] When the robot arrives at its destination, it sends an arrival notification to the server.

[1882] The server notifies the user terminal of the arrival information, and the series of movements is completed.

[1883] Input: Arrival notification from the robot

[1884] Output: Arrival information notification to user terminal

[1885] Prompt example

[1886] Possible input prompts for a generative AI model include:

[1887] "Please explain the entire process from when a user enters a pickup request until the nearest robot comes to pick them up."

[1888] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1889] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions, and is realized by linking the user terminal, server, robot, electric senior car, and emotion engine.

[1890] First, the resident (hereafter referred to as the user) uses a user device such as a smartphone or tablet. A dedicated application is installed on the user device, and the user inputs a pickup request through this application. After logging in to the application, the user selects a room in their home as the destination and sends the pickup request.

[1891] Next, there is a server that receives requests sent from user devices. The server analyzes the requests received from the user and sends instructions to the nearest autonomous mobile security robot (hereafter referred to as the robot) based on the user's location and destination information. The server monitors the robot's location and status in real time, and sends control signals as necessary to remotely control the robot's operation.

[1892] The robot receives instructions from the server and heads to the designated entrance lobby to connect the electric senior car. The robot is equipped with high-precision sensors that allow it to detect and avoid obstacles. It is also capable of smooth communication with residents using AI technology. It is also equipped with an emotion engine that can analyze the user's emotional state and respond appropriately. The emotion engine recognizes emotions based on the user's facial expressions, voice, and behavioral data, and the robot responds appropriately.

[1893] The specific operation of the system will be described below.

[1894] Consider a scenario where Resident A goes out and wants to return to his room at home.

[1895] 1. User (A):

[1896] Open the smartphone app and enter your pickup request.

[1897] 2. Device (app):

[1898] A request for transportation from Mr. A is sent to the server.

[1899] 3. Server:

[1900] Receive the request and confirm Mr. A's location and destination.

[1901] It instructs the nearest robot to move and perform a series of actions.

[1902] 4. Robots:

[1903] After receiving instructions from the server, the person heads to the entrance lobby and connects the senior car.

[1904] The system uses an emotion engine to analyze Mr. A's emotional state and greets him appropriately upon arrival.

[1905] 5. User (A):

[1906] Check the senior car connected to the robot in the entrance lobby and get in.

[1907] Travel with peace of mind thanks to the robot's voice guidance.

[1908] 6. Robots:

[1909] It uses sensors to avoid obstacles and safely drive towards your room in the home.

[1910] If Mr. A seems anxious while driving, the emotion engine will detect this and reassure him with appropriate words.

[1911] 7. Terminal:

[1912] The robot's location is displayed in real time and provided to Mr. A.

[1913] 8. Robots:

[1914] When they arrive at their destination, they inform Mr. A of their arrival and greet him in a friendly manner.

[1915] 9. User (A):

[1916] I get off the senior car and enter my room at home.

[1917] 10. Robots:

[1918] The server is notified that the transfer is complete, and the emotional state of Mr. A recognized by the emotion engine is also sent.

[1919] If there is no next task, move to an appropriate waiting position.

[1920] This series of processes will realize a system that allows users to move around the apartment building safely and efficiently while taking their emotions into consideration. The emotion engine's response is expected to provide an elderly-friendly environment and improve the quality of life.

[1921] The processing flow will be explained below.

[1922] Step 1:

[1923] The user launches the app on their smartphone and logs in. Once logged in, the transfer request input screen is displayed.

[1924] Step 2:

[1925] On the pickup request input screen, the user selects "room at home" as the destination and enters the pickup request. Then, the user presses the "Submit" button to confirm the request.

[1926] Step 3:

[1927] The device transmits the location information along with the pickup request entered by the user to the server, including the user's destination information.

[1928] Step 4:

[1929] The server receives the request and analyzes the user's location and destination information, based on which it identifies the nearest autonomous mobile security robot.

[1930] Step 5:

[1931] The server sends a control signal to the identified robot to instruct it to move to the entrance lobby and connect the electric senior car.

[1932] Step 6:

[1933] The robot receives instructions from the server, heads to the designated entrance lobby, and connects the electric senior car.

[1934] Step 7:

[1935] When the robot arrives at the entrance lobby, it notifies the user of its arrival with a voice prompt. It also uses a built-in emotion engine to analyze the user's emotions and provides voice prompts and greetings according to the user's emotional state.

[1936] Step 8:

[1937] The user heads to the entrance lobby and sees the electric senior car connected to the robot.

[1938] Step 9:

[1939] The user gets into the senior car, checks for safety, and once they're done, presses the "Depart" button on the app.

[1940] Step 10:

[1941] The terminal transmits the user's "depart" instruction to the server.

[1942] Step 11:

[1943] The server sends a "go" command to the robot.

[1944] Step 12:

[1945] The robot receives the "depart" command and begins autonomously moving toward the room in the home. It uses built-in sensors to detect obstacles and drive safely. It also uses an emotion engine to monitor the user's emotions.

[1946] Step 13:

[1947] The robot analyzes the user's emotions while moving, and if it detects anxiety or tension, it will provide voice guidance and dialogue to provide comfort.

[1948] Step 14:

[1949] The terminal displays the robot's location information in real time and provides it to the user.

[1950] Step 15:

[1951] When the robot arrives at its destination, it notifies the user and speaks in a friendly manner. Based on the analysis results of its emotion engine, it checks whether the user is calm.

[1952] Step 16:

[1953] The user gets out of the mobility scooter and enters a room in their home.

[1954] Step 17:

[1955] The robot notifies the server that the transfer is complete and also sends the user's emotional state as recognized by the emotion engine.

[1956] Step 18:

[1957] The server marks the robot's task as completed and checks to see if there are any other tasks waiting.

[1958] Step 19:

[1959] If there is no next task, the robot moves to an appropriate waiting position and waits for the next instruction.

[1960] This series of processes will realize a system that utilizes an emotion engine to allow elderly people to move safely around an apartment building while also taking into consideration their emotional state.

[1961] Example 2

[1962] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1963] Conventional outing support systems simply support movement without considering the user's emotional state. As a result, elderly and emotionally unstable users often feel anxious while out and about. Another issue is that communication with the user is not smooth, making it difficult for the user to feel familiar with the system.

[1964] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a user terminal operated by the resident to provide outing assistance, means for receiving and managing requests sent from the user terminal, means including a robot that moves autonomously based on instructions sent from the server and connects to an electric senior car, an electric senior car for transporting the resident while connected to the robot, means for detecting obstacles using sensors mounted on the robot and driving safely, means for remotely controlling the operation of the robot using control signals sent from the server, means for analyzing the emotional state of the resident using an emotion engine, and means for providing appropriate greetings or calls based on the user's emotions. This enables safe and friendly outing assistance that takes the user's emotional state into consideration.

[1965] A "user terminal" is an electronic device operated by a user, and includes a smartphone, a tablet, and the like.

[1966] A "server" is a computer system that receives and manages requests sent from user terminals.

[1967] A "request" is request information sent from a user terminal to a server, and includes instructions for transportation or movement.

[1968] The "robot" is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car.

[1969] An "electric senior car" is an electric mobility device that is connected to a robot to safely transport residents.

[1970] "Sensors" are devices installed on robots to detect obstacles, and include LIDAR and cameras.

[1971] A "control signal" is an instruction sent from the server to the robot to remotely control the robot's operations.

[1972] An "emotion engine" is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response.

[1973] "Appropriate greetings and calls" are communication actions that the robot takes toward the user based on the analysis results of the emotion engine.

[1974] This invention is an outing support system that uses an autonomous mobile robot and an electric senior car combined with an emotion engine that recognizes the user's emotions. The main components of the system are a user terminal, a server, a robot, an electric senior car, and an emotion engine, all of which work in conjunction with each other.

[1975] User terminal

[1976] A user terminal is an electronic device operated by a user, and includes smartphones, tablets, etc. A dedicated application is installed on the device, and the user uses this application to input a pickup request. For example, simply inputting a request such as "I want to go to my room at home" initiates a series of operations in the system.

[1977] server

[1978] The server is a computer system that receives and manages requests sent from the user's terminal. The server analyzes the request data and sends appropriate instructions to the robot based on the user's current location and destination. During this process, the server constantly monitors the robot's location and status in real time, and sends control signals each time to remotely control the robot's operation.

[1979] robot

[1980] The robot is an autonomous mobile device that operates autonomously based on instructions from a server and is connected to an electric senior car. The robot is equipped with high-precision sensors and can move safely while detecting obstacles using LIDAR and cameras. The robot also has an emotion engine that recognizes emotions based on the user's facial expressions, voice, and behavioral data. For example, if the user appears anxious, the robot can reassure them with appropriate words such as, "Don't worry. We'll soon arrive at our destination."

[1981] Electric senior car

[1982] An electric senior car is an electric mobility device that connects to a robot to safely transport residents. Because it is controlled by the robot, the user does not need to drive it themselves. It allows residents to travel safely and smoothly to specific destinations within the apartment complex.

[1983] Emotion Engine

[1984] The emotion engine is an artificial intelligence technology that analyzes the user's emotional state and determines the robot's response. The emotion engine uses data obtained from a camera and microphone to analyze the user's facial expressions and tone of voice, recognizing their emotional state in real time. This allows the robot to respond appropriately to the user, providing a sense of security, especially for elderly and emotionally unstable users.

[1985] Add specific examples to the description

[1986] For example, if resident A wants to return to their room after being out, the following scenario can be considered: First, resident A opens the smartphone app and enters a request for transportation. The app sends the request to the server, which receives and analyzes the request, and then sends instructions to the nearest robot. Based on instructions from the server, the robot heads to the entrance lobby and connects its electric senior car. The robot then uses its emotion engine to analyze resident A's emotional state and responds with an appropriate greeting, such as, "Hello, resident A. How are you feeling today?" resident A gets into the senior car and follows the robot's guidance, feeling reassured as they travel. Finally, when the robot arrives at its destination and resident A gets out of the senior car, the robot notifies the server that transportation is complete and waits for the next task.

[1987] Prompt Sentence Examples

[1988] Below are some example prompts for a generative AI model:

[1989] "Please explain how an autonomous mobile robot equipped with an emotion engine can analyze the emotional state of its user."

[1990] "Please explain in detail the procedure for welcoming a robot using the emotion engine."

[1991] This not only allows users to use the system with peace of mind, but also provides an environment that is particularly friendly to elderly people.The present invention aims to improve the quality of life of users.

[1992] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1993] Step 1:

[1994] The user opens a dedicated app installed on their smartphone or tablet and enters a transportation request.

[1995] Specific operation: The user launches the app, enters their destination (e.g., their room number at home), and presses the pick-up button.

[1996] Input: Trip request information entered by the user (destination, current location).

[1997] Output: Trip request data generated within the app.

[1998] Step 2:

[1999] The terminal sends the user's transportation request to the server.

[2000] Specific behavior: The app sends a request containing the user's location information (GPS data) and destination information to the server as an HTTP POST request.

[2001] Input: The generated trip request data.

[2002] Output: The request data sent to the server.

[2003] Step 3:

[2004] The server receives the request and verifies the user's location and destination.

[2005] Specific operation: The server analyzes the request data and identifies the user's current location (for example, the entrance to an apartment building).

[2006] Input: Request data sent from the device.

[2007] Output: Parsed user location and destination information.

[2008] Step 4:

[2009] The server instructs the nearest robot to move and perform a series of actions.

[2010] Specific operation: The server checks the robot's current location, calculates the optimal route, and then sends instructions such as "head to the entrance, connect the senior cart, and welcome the user."

[2011] Input: User's current location, destination, and robot's current location data.

[2012] Output: Movement and motion instruction data sent to the robot.

[2013] Step 5:

[2014] The robot receives instructions from the server, heads to the entrance lobby, and connects the electric senior car.

[2015] Specific operation: The robot uses its own navigation system to autonomously navigate to the entrance lobby and physically connect with the senior car.

[2016] Input: Movement and action instruction data sent from the server.

[2017] Output: Arrival at the entrance lobby and the connection status of the mobility scooter.

[2018] Step 6:

[2019] The robot uses an emotion engine to analyze the user's emotional state and greets them appropriately upon arrival.

[2020] Specific operation: The robot uses a camera and microphone to analyze the user's facial expressions and tone of voice, and then greets them with a greeting such as, "Hello, Mr. A. How are you feeling today?"

[2021] Input: User's facial expression data, voice data.

[2022] Output: Analysis results and a voice message based on them.

[2023] Step 7:

[2024] The user checks the senior car connected to the robot in the entrance lobby and gets in.

[2025] Specific operation: The user follows the robot's instructions to get into the senior car.

[2026] Input: User confirmation and ride operation.

[2027] Output: Riding in a mobility scooter.

[2028] Step 8:

[2029] The robot's voice guidance provides users with a sense of security as they move around.

[2030] What it does: The robot provides voice guidance such as, "Don't worry, we're on our way to your next destination."

[2031] Input: Robot voice guidance data.

[2032] Output: A voice message to the user.

[2033] Step 9:

[2034] The robot uses sensors to avoid obstacles and safely navigates towards your room.

[2035] How it works: The robot uses a LIDAR sensor to scan the surrounding environment and automatically selects a safe route to move.

[2036] Input: Sensor data, environmental data.

[2037] Output: Obstacle-avoiding route information and safe movement status.

[2038] Step 10:

[2039] If the user appears anxious while the robot is driving, the emotion engine will detect this and provide appropriate reassurance.

[2040] What it does: The emotion engine detects the user's anxiety and provides reassuring words such as, "You're almost there."

[2041] Input: User's facial expression data, voice data.

[2042] Output: Analysis results and a voice message based on them.

[2043] Step 11:

[2044] The robot arrives at the destination and notifies the user of its arrival while speaking in a friendly manner.

[2045] Specific operation: The robot will provide guidance such as, "Mr. A, you have arrived at your destination. Thank you for your hard work."

[2046] Input: Destination arrival data.

[2047] Output: A voice message to the user.

[2048] Step 12:

[2049] The user gets out of the mobility scooter and enters a room in their home.

[2050] Specific actions: The user gets out of the mobility scooter, unlocks the door to their home, and enters.

[2051] Input: User exit and entry operations.

[2052] Output: The user's room entry status.

[2053] Step 13:

[2054] The robot notifies the server that the transfer is complete and also transmits the user's emotional state as recognized by the emotion engine.

[2055] Specific operation: The robot uploads the data of the completed transfer to the server and also sends the emotion data.

[2056] Input: Pick-up and drop-off completion data, emotional state data.

[2057] Output: Data sent to the server.

[2058] Step 14:

[2059] If the robot has no next task, it moves to an appropriate waiting position.

[2060] Specific behavior: The robot moves to the waiting station and waits for further instructions.

[2061] Input: Next task availability data.

[2062] Output: Waiting in standby position.

[2063] (Application example 2)

[2064] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2065] There is a need for systems that allow elderly people and people who require physical assistance to shop safely and with peace of mind in physical stores. It is also important to provide appropriate support according to their emotional state. Existing systems have had problems with insufficient emotion recognition and are unable to adequately respond to the anxiety and confusion of elderly people.

[2066] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and managing requests sent from a user terminal, means for controlling an autonomous mobile device that moves autonomously based on instructions sent from the server and connects to an electric vehicle, and means for recognizing the emotional state of residents using an emotion engine installed in the autonomous mobile device and taking appropriate action. This enables elderly people and people who require physical assistance to move safely within a store and shop with peace of mind. Furthermore, the emotion engine provides appropriate support according to the user's emotional state, thereby improving their sense of security and satisfaction.

[2067] "Residents" are users of the system, primarily elderly people and those who require physical assistance.

[2068] "User terminal" refers to electronic devices such as smartphones and tablets that are operated by residents.

[2069] A "request" refers to information sent from a user terminal to a server to request support or movement.

[2070] The term "server" refers to a computer system that manages requests sent from user terminals and sends instructions to autonomous mobile devices.

[2071] "Autonomous mobile equipment" refers to robots and similar equipment that move autonomously and connect to electric vehicles based on instructions from a server to support residents.

[2072] "Electric vehicle" refers to an electric senior car or other electrically powered vehicle that is connected to an autonomous mobility device and used to transport residents.

[2073] "Sensor" refers to a device installed on an autonomous mobile device to detect obstacles in the surrounding area.

[2074] "Emotion engine" refers to artificial intelligence technology that recognizes residents' emotional state and responds appropriately.

[2075] "Artificial intelligence technology" refers to technology used by autonomous mobile devices to facilitate communication with residents.

[2076] This invention is a system that helps elderly people and people who require physical assistance to shop safely in physical stores. The system is composed of components including a server, user terminals, autonomous mobile devices, electric vehicles, sensors, an emotion engine, and artificial intelligence technology.

[2077] First, the user device is a smartphone or tablet. Residents submit support requests through a dedicated application. This request is sent from the device to a server. The server receives the request and analyzes the resident's location and the type of support required. Based on the analysis results, it sends instructions to the nearest autonomous mobile device.

[2078] The autonomous mobile device receives instructions from the server and moves to a designated location within the store. This device connects to an electric vehicle (such as an electric cart or senior cart) to transport the resident. The autonomous mobile device is equipped with sensors that detect and avoid obstacles in the surrounding area. It is also equipped with an emotion engine that analyzes the resident's facial expressions, voice, and behavior to recognize their emotional state and respond appropriately.

[2079] The server monitors the location of the autonomous mobile device in real time and remotely controls it as needed, using a high-precision control system.

[2080] As a specific example, consider the following scenario.

[2081] Scenario: Elderly person B goes shopping at a supermarket

[2082] 1. User (Mr. B):

[2083] Open the smartphone app, select "Shopping Support," and submit a support request.

[2084] 2. Device (app):

[2085] Send Mr. B's shopping support request to the server.

[2086] 3. Server:

[2087] Receive the request and confirm Mr. B's location and the support he needs.

[2088] Send instructions to the nearest autonomous mobile device.

[2089] 4. Autonomous mobile devices:

[2090] Following the server's instructions, I headed towards the entrance of the supermarket.

[2091] The emotional engine is used to analyze Mr. B's emotional state and provide an appropriate greeting or greeting upon arrival.

[2092] 5. Person B:

[2093] The user contacts an autonomous mobile device near the entrance and requests assistance with shopping.

[2094] Follow the voice guidance of the autonomous mobile device and start shopping with peace of mind.

[2095] 6. Autonomous mobile devices:

[2096] As they move around the store together, they guide B to products based on his shopping list.

[2097] Move safely while detecting obstacles.

[2098] If Person B feels anxious, the emotion engine will detect this and provide appropriate words to reassure them.

[2099] Hardware and Software Used

[2100] User device: Smartphone (iOS / Android)

[2101] Server: Cloud server (AWS or Google Cloud) + Flask (Python web server framework)

[2102] Autonomous mobile device: Autonomous mobile robot (equipped with LiDAR sensor, camera, and microphone)

[2103] Emotion Engine: Emotion Recognition Model with TensorFlow / Keras

[2104] An example prompt is:

[2105] "Imagine that Person B goes to the supermarket and asks an autonomous mobile device to help him with his shopping. Person B is feeling a little anxious. How would the autonomous mobile device respond to reassure Person B?"

[2106] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2107] Step 1:

[2108] User (Mr. B)

[2109] Open the smartphone app, select "Shopping Support," and submit a support request.

[2110] Input: B's location information, request details

[2111] Output: Support request data

[2112] Step 2:

[2113] Device (app)

[2114] Send Mr. B's shopping support request to the server.

[2115] Input: Support request data

[2116] Output: Request data sent to the server

[2117] Step 3:

[2118] server

[2119] The system receives the request, checks B's location and the support required, analyzes the request data, and sends instructions to the nearest autonomous mobile device.

[2120] Input: Request data sent to the server

[2121] Data processing / calculation: Analyzes the request and selects the most suitable autonomous mobile device

[2122] Output: Instruction data sent to the autonomous mobile device

[2123] Step 4:

[2124] Autonomous mobile equipment

[2125] It receives instructions from the server and moves to a designated location within the store. During its movement, it uses sensors to detect surrounding obstacles and proceed safely.

[2126] Input: Instruction data sent to the autonomous mobile device

[2127] Data processing / calculation: Calculating movement routes based on location information, detecting obstacles using sensors

[2128] Output: Action instructions to move safely

[2129] Step 5:

[2130] Autonomous mobile equipment

[2131] Upon arrival, the emotion engine is used to analyze Mr. B's emotional state and an appropriate greeting or greeting is given.

[2132] Input: Person B's facial expression and voice data

[2133] Data processing / calculation: Emotion recognition using TensorFlow / Keras, and decision on how to respond based on the analysis results

[2134] Output: An appropriate response to Person B

[2135] Step 6:

[2136] User (Mr. B)

[2137] Users contact an autonomous mobile device near the entrance and request shopping assistance. They then follow the audio guidance provided by the autonomous mobile device and begin shopping with peace of mind.

[2138] Input: Autonomous mobile device information

[2139] Output: Person B's actions

[2140] Step 7:

[2141] Autonomous mobile equipment

[2142] As the robot moves around the store together with B, it guides him to products based on his shopping list. Along the way, it again uses sensors to detect and avoid obstacles. If B feels confused or anxious, the emotion engine detects this and uses appropriate words to reassure him.

[2143] Input: B's shopping list, facial expressions, voice data, sensor information

[2144] Data processing / calculation: Navigation to product locations based on shopping lists, emotion recognition, obstacle avoidance

[2145] Output: Product guidance, obstacle avoidance behavior, appropriate response

[2146] Step 8:

[2147] Autonomous mobile equipment

[2148] Once the customer has finished shopping, they will be guided to the cash register and supported until payment is complete.

[2149] Input: B's actions, cash register location information

[2150] Data processing / calculation: Secure route calculation, payment completion confirmation

[2151] Output: Instructions to cashier, necessary responses

[2152] Step 9:

[2153] server

[2154] It notifies the robot that the shopping support is complete and records the emotional state of the robot according to the emotion engine. If there is no next task, the robot moves to an appropriate waiting position.

[2155] Input: Completion notification from autonomous mobile device, emotional state data

[2156] Data processing / calculation: Confirmation of task completion, movement plan to waiting position

[2157] Output: Updates to system logs, instructions to move autonomous mobile devices to standby positions

[2158] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2159] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2160] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2161] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2162] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2163] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2164] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2165] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2166] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2167] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2168] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2169] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2170] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2171] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2172] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2173] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2174] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2175] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2176] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2177] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2178] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2179] The following is further disclosed regarding the above embodiment.

[2180] (Claim 1)

[2181] a user terminal operated by a resident to provide assistance with going out;

[2182] a server that receives and manages requests sent from user terminals;

[2183] A robot that moves autonomously based on instructions sent from a server and connects to an electric senior car;

[2184] An electric senior car that connects to the robot to transport residents,

[2185] A means to detect obstacles using sensors installed on the robot and drive safely;

[2186] a means for remotely controlling the operation of the robot by a control signal transmitted from the server;

[2187] A system including:

[2188] (Claim 2)

[2189] The system according to claim 1, further comprising: a means for smoothly communicating with residents using AI technology installed in the robot.

[2190] (Claim 3)

[2191] 10. The system of claim 1, further comprising: means for a resident to input a request through an application installed on a user terminal; and means for providing real-time location information of the robot.

[2192] "Example 1"

[2193] (Claim 1)

[2194] an information processing device operated by a user for outing assistance;

[2195] a data processing device that receives and manages requests transmitted from an information processing device;

[2196] a mechanical device that moves autonomously based on instructions transmitted from a data processing device and connects to the mobile device;

[2197] a moving device for moving a user while being connected to the mechanical device;

[2198] A means for detecting obstacles using sensors mounted on the machinery and driving safely;

[2199] means for remotely controlling the operation of the machine by control signals transmitted from the data processing device;

[2200] data analysis means for analyzing requests from the information processing device and issuing appropriate instructions;

[2201] a mechanical means for moving the mechanical device in response to an instruction and for connecting the moving device at a designated point;

[2202] A system including:

[2203] (Claim 2)

[2204] 10. The system of claim 1, further comprising means for facilitating interaction with a user using artificial intelligence technology installed in the mechanical device.

[2205] (Claim 3)

[2206] a means for a user to input a request using an application installed in the information processing device;

[2207] a means for providing real-time machine location information;

[2208] a means for notifying the completion of the transfer when the machine arrives at the designated location;

[2209] 10. The system of claim 1, comprising:

[2210] "Application Example 1"

[2211] (Claim 1)

[2212] a user terminal operated by a resident to provide assistance with going out;

[2213] a server that receives and manages requests sent from user terminals;

[2214] A robot that moves autonomously based on instructions sent from a server and connects to an electric senior car;

[2215] An electric senior car that connects to the robot to transport residents,

[2216] A means to detect obstacles using sensors installed on the robot and drive safely;

[2217] a means for remotely controlling the operation of the robot by a control signal transmitted from the server;

[2218] A means of visualizing the robot's position information and movement path in real time using smart glasses or a head-mounted display;

[2219] A system including:

[2220] (Claim 2)

[2221] A means of smooth communication with residents using AI technology installed in the robot,

[2222] and a means for displaying the position information and movement path of the robot on smart glasses or a head-mounted display.

[2223] 10. The system of claim 1.

[2224] (Claim 3)

[2225] a means for residents to input requests by means of an application installed on a user terminal;

[2226] a means for providing real-time location information of the robot;

[2227] A means to display the robot's location information in conjunction with smart glasses or a head-mounted display,

[2228] 10. The system of claim 1, comprising:

[2229] "Example 2: Combining Emotion Engines"

[2230] (Claim 1)

[2231] a user terminal operated by a resident to provide assistance with going out;

[2232] a server that receives and manages requests sent from user terminals;

[2233] A robot that moves autonomously based on instructions sent from a server and connects to an electric senior car;

[2234] An electric senior car that connects to the robot to transport residents,

[2235] A means to detect obstacles using sensors installed on the robot and drive safely;

[2236] a means for remotely controlling the operation of the robot by a control signal transmitted from the server;

[2237] A means of analyzing the emotional state of residents by using an emotion engine;

[2238] a means for making appropriate greetings or calls based on the user's emotions;

[2239] A system including:

[2240] (Claim 2)

[2241] 2. The system of claim 1, further comprising: means for smoothly communicating with residents using AI technology installed in the robot; and means for analyzing and responding to emotional states using an emotion engine.

[2242] (Claim 3)

[2243] The system of claim 1, further comprising: means for residents to input requests through an application installed on a user terminal; means for providing real-time location information of the robot; and means for sensing the emotional state of residents while driving and providing appropriate reassurance.

[2244] "Application example 2 when combining emotion engines"

[2245] (Claim 1)

[2246] A terminal operated by residents to provide support,

[2247] a server that receives and manages requests sent from user terminals;

[2248] an autonomous mobile device that moves autonomously based on instructions transmitted from a server and connects to an electric vehicle;

[2249] an electric vehicle for transporting residents while connected to the autonomous mobile device;

[2250] A means for detecting obstacles using sensors mounted on the autonomous mobile device and driving safely;

[2251] a means for remotely controlling the operation of the autonomous mobile device by a control signal transmitted from the server;

[2252] A means for recognizing the emotional state of residents using an emotion engine installed in the autonomous mobile device and taking appropriate action;

[2253] A system including:

[2254] (Claim 2)

[2255] The system according to claim 1, further comprising: means for smoothly communicating with residents using artificial intelligence technology installed in the autonomous mobile device.

[2256] (Claim 3)

[2257] 10. The system of claim 1, further comprising: means for a resident to input a request by an application installed on a user terminal; means for providing location information of the autonomous mobile device in real time; and means for transmitting an emotional state sensed by the emotion engine to a server. [Explanation of symbols]

[2258] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a user terminal operated by a resident to provide assistance with going out; a server that receives and manages requests sent from user terminals; A robot that moves autonomously based on instructions sent from a server and connects to an electric senior car; An electric senior car that connects to the robot to transport residents, A means to detect obstacles using sensors installed on the robot and drive safely; a means for remotely controlling the operation of the robot by a control signal transmitted from the server; A system including:

2. The system according to claim 1, further comprising a means for smoothly communicating with residents using AI technology installed in the robot.

3. The system according to claim 1 , further comprising: means for a resident to input a request through an application installed on a user terminal; and means for providing real-time location information of the robot.

Images