Evacuation guidance system using autonomous operation vehicle

Abstract

To provide an evacuation guidance system using an autonomous operation vehicle.SOLUTION: An evacuation guidance system 100 using an autonomous operation vehicle includes an operation server 30 for an autonomous operation vehicle 20 which includes a memory unit 32 in which three-dimensional map data related to roads, surrounding environment, facility equipment including an elevators 41, and a building 40 equipped with a fire alarm system 52 is registered, a fire alarm system connected to the operation server via a network 50, an autonomous operation vehicle which is connected to the operation server via a network and includes an audio output unit 25, and a facility equipment control unit 48 which is connected to the operation server via a network and controls the facility equipment on the basis of facility equipment usage information 48a and facility equipment control information 48b from the autonomous operation vehicle. When receiving a fire information signal from a receiver, the operation server displays the location of the fire on map data for the facility, compares it with pre-created evacuation routes, creates a safe evacuation route and transmits it to the autonomous operation vehicle, and the control unit of the autonomous operation vehicle is configured to output audio evacuation guidance to people in the vicinity.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an evacuation guidance system using an autonomous vehicle in which evacuation routes are created in advance for apartment buildings, office buildings, hospitals, factories, underground parking lots, stadiums, convention centers, and various other facilities (hereinafter collectively referred to as structures or simply buildings) by referring to map data, and in the event of a fire, the fire alarm equipment and the autonomous vehicle's operation server work together to drive the autonomous vehicle along the evacuation route, providing evacuation guidance and moving it to a safety confirmation position. [Background technology]

[0002] As an example of an autonomous vehicle operation system, Patent Document 1 discloses an autonomous vehicle that travels along a set route, which is equipped with a pair of left and right display units on the front of the main body and a monitoring camera that captures images in front, on the sides, and behind the autonomous vehicle, and which displays a predetermined eye-shaped image on the display units based on the state of the autonomous vehicle and / or images captured by the monitoring camera, thereby changing the eye-shaped image depending on the status of obstacles, making the autonomous vehicle more approachable to stores, delivery destinations, and people around it while traveling. This autonomous vehicle is capable of communicating with surrounding pedestrians and the like while traveling, and is expected to travel smoothly.

[0003] It is also known to provide an audio output unit in an autonomous vehicle and to notify the autonomous vehicle of its driving status by outputting information about the vehicle's driving status, such as its direction of travel, from the audio output unit while the vehicle is driving.

[0004] Furthermore, a self-driving vehicle operation system is also known in which, for example, map data including the internal passageways and elevators of a building equipped with elevators is prepared, and the self-driving vehicle can refer to this map data to use the internal passageways and elevators of the building (see Patent Document 3). In this case, a system has already been put into practical use in which an audio output unit of the self-driving vehicle similarly outputs audio to notify pedestrians in the internal passageways and elevator users of the presence and running status of the self-driving vehicle. A self-driving vehicle that moves along markers placed on road surfaces or walls is also known (Patent Document 2).

[0005] Furthermore, Patent Document 4 discloses a disaster prevention cooperation system in which a fire alarm system and a device management system that manages work robots are linked together. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Application Publication No. WO2020 / 145189 A1 [Patent Document 2] Republished Patent Publication WO2020 / 013337 [Patent Document 3] International Application Publication No. WO2023 / 085136 A1 [Patent Document 4] Japanese Patent Application Publication No. 2017-215625 [Patent Document 5] Japanese Patent Application Publication No. 2023-46298 Summary of the Invention [Problem to be solved by the invention]

[0007] Recently, the use of robots for delivery, cleaning, security, etc. using self-driving vehicles has been progressing, but there are challenges in considering the use of robots to guide evacuation routes for people in buildings and other facilities in the event of a fire, earthquake, etc.

[0008] The present invention aims to provide an evacuation guidance system using an autonomous vehicle that creates evacuation routes in advance by referring to map data, and in the event of a fire inside a structure such as a building, the fire alarm equipment and the autonomous vehicle's operation server work together to drive the autonomous vehicle along the evacuation route, guiding people inside the structure to safety and moving it to a safety confirmation position. [Means for solving the problem]

[0009] The evacuation guidance system using an autonomous driving vehicle of the present invention includes: an autonomous vehicle operation server having a memory unit in which three-dimensional map data relating to roads, surrounding environments, facility equipment including elevators, and buildings equipped with fire alarm systems is registered; the fire alarm system connected to the operation server via a network; the autonomously driven vehicle connected to the operation server via the network; a facility equipment control unit connected to the operation server via the network and controlling the facility equipment based on facility equipment usage information and facility equipment control information from the autonomously driven vehicle, the autonomous driving vehicle comprises a drive unit, a control unit, an imaging unit, a detection unit consisting of an attitude detection sensor and a position sensor, a transmission / reception unit connected to the network, and an audio output unit; the operation server stores in the storage unit an evacuation route that has been created in advance based on the map data; The fire alarm system outputs a fire alarm when a fire signal from a fire detector installed in the building is received by a receiver, and outputs a fire information signal including the location of the fire to the operation server; When the operation server receives the fire information signal from the fire alarm system, it displays the location of the fire on map data of the facility, compares it with a pre-created evacuation route, creates an evacuation route including a safe elevator, and transmits the created evacuation route to the autonomously driven vehicle; The control unit of the autonomous vehicle is characterized in that it outputs evacuation guidance audio to people in the vicinity from the audio output unit.

[0010] In the above configuration, the facility equipment preferably includes an opening and closing device such as a security door, a gate, an automatic door, or a shutter, and the elevator. Preferably, the autonomous vehicle detects operation information of facility equipment including the elevator while moving from the current position to the safety confirmation position, and sends the operation information to the operation server. Preferably, while the autonomous vehicle is moving from the current position to the safety confirmation position, a control unit of the autonomous vehicle controls getting on and off the elevator and opening and closing of the opening and closing device. The control unit of the autonomous vehicle preferably outputs an announcement from an audio output unit of the autonomous vehicle to notify people in the vicinity of the evacuation guidance of the autonomous vehicle while the autonomous vehicle is traveling along the evacuation route, and outputs an announcement from the audio output unit of the autonomous vehicle regarding the use of facility equipment including the elevator based on usage information of the facility equipment including the elevator. The operation server preferably remotely monitors and / or remotely controls the operation of the autonomous vehicles based on the current location of the autonomous vehicles and evacuation routes, as well as the relative positions of the autonomous vehicles with each other or the autonomous vehicles and the surrounding environment, etc. The building preferably includes a management server that controls the entire building, and the management server controls the control unit of the facility equipment including the elevator based on usage information of the facility equipment including the elevator. The imaging unit preferably includes a camera or LIDAR capable of length measurement, a computer that processes image signals acquired by the camera or LIDAR, and a storage device, and acquires AI analysis information based on image signals including fire, smoke, and people acquired along the evacuation route, and outputs the AI ​​analysis information to a control unit of facility equipment including elevators and to an operation server.The imaging unit preferably includes an infrared camera, a computer that processes image signals acquired by the infrared camera, and a storage device, and acquires AI analysis information based on image signals including fire, smoke, and people acquired along the evacuation route, and outputs the AI ​​analysis information to a control unit of facility equipment including elevators and to the operation server. The operation server preferably receives the AI ​​analysis information, modifies the evacuation route as necessary, and sends the modified evacuation route information to one or more autonomous vehicles in the building and / or surrounding environment. The operation server preferably uses AI analysis to count the number of occupants in elevators, etc., based on images acquired by the imaging unit of the autonomous vehicle or a surveillance camera installed in the building, and makes decisions including adjusting and resetting the evacuation route. When the operation server receives elevator stop information due to an earthquake from the elevator control unit and / or a management server that controls the building, it preferably transmits the elevator stop information to the autonomous vehicle and creates an evacuation route so that the autonomous vehicle and surrounding people can exit the elevator at the nearest floor to where the elevator will stop, and provides evacuation guidance to notify people around the autonomous vehicle of the occurrence of an earthquake and the suspension of elevator operation. The autonomous vehicle preferably further includes a display unit that displays evacuation guidance based on the evacuation guidance information from the operation server and / or the building's management server. The operation server preferably includes a self-location estimation unit that determines the current location of a person from image data transmitted from a mobile device carried by the person, and when it receives a fire information signal, it transmits the estimated current location and evacuation route of the person to the mobile device, and the control unit of the mobile device displays the current location and evacuation route via audio output and / or display unit. Preferably, the autonomous vehicle transmits images acquired by the imaging unit while traveling during normal operations or images acquired by a surveillance camera installed in the building to the operation server, and the operation server performs AI analysis of the acquired images. If an obstacle that impedes passage along the evacuation route or facility equipment is detected, the operation server sends an alert to the management server of the building. The self-driving vehicle is preferably a vehicle for various work tasks such as delivery, security, cleaning, disinfection, reception and guidance, and advertising, a passenger vehicle for transporting users such as the elderly, injured or disabled, or an autonomous delivery vehicle for delivering various items such as food, drink, and merchandise.

[0011] According to the present invention, an evacuation guidance system using an autonomous vehicle can be provided in which an evacuation route is created in advance by referring to map data, and in the event of a fire occurring within a building, the fire alarm equipment and the autonomous vehicle's operation server work together to drive the autonomous vehicle along the evacuation route, guiding people within the building to safety and moving it to a safety confirmation position. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing the configuration of a first embodiment of an evacuation guidance system using an autonomously driven vehicle according to the present invention; [Figure 2] 2A and 2B show an example of the external configuration of the autonomous driving vehicle of FIG. 1, in which (A) is a front perspective view and (B) is a rear perspective view. [Figure 3] FIG. 3 is a block diagram showing the internal configuration of the autonomous driving vehicle of FIG. 2. [Figure 4] FIG. 2 is a schematic diagram showing the configuration of an elevator hall. [Figure 5] 2 is a flowchart showing the procedure for automatic driving of the automatically driven vehicle of FIG. 1. [Figure 6] 2 is a flowchart showing the operation of the autonomously driven vehicle of FIG. 1 when using an elevator. [Figure 7] 2 is a flowchart showing the procedure for evacuation guidance of an autonomously driven vehicle in the evacuation guidance system for an autonomously driven vehicle of FIG. 1. [Figure 8] 1 is a block diagram showing the configuration of an evacuation guidance system that provides evacuation guidance using mobile terminals carried by people inside a building. [Figure 9] FIG. 10 is a workflow diagram of data set generation in the operation server. [Figure 10] FIG. 10 is a workflow diagram of the operation server estimating the user's own position. [Figure 11] This figure shows a two-dimensional photograph taken by a user that serves as a search image, and a two-dimensional image extracted by feature matching from a three-dimensional point cloud on the operation server. [Figure 12] 10 is a photograph showing an example of an AR navigation display. DETAILED DESCRIPTION OF THE INVENTION

[0013] (First embodiment) The present invention will be described in detail below based on several embodiments shown in the drawings. However, each embodiment is merely for the purpose of explaining the present invention, and the present invention is not limited thereto. In the illustrated examples, the self-driving vehicle is described as a one-box type automated delivery vehicle that transports ordered food and beverages or purchased goods. However, the shape and structure of the self-driving vehicle are not limited to these vehicles as long as it can enter and exit buildings and elevators. Furthermore, the present invention can be applied not only to delivery vehicles for goods, but also to various automated work vehicles performing security, cleaning, disinfection, monitoring, reception guidance, advertising (signage), etc., or to automated passenger vehicles transporting users such as the elderly, injured, or physically disabled. Buildings in which elevators are installed are not limited to office buildings or commercial facilities, but may also be large, high-rise apartment buildings.

[0014] FIG. 1 shows the overall configuration of a first embodiment of an evacuation guidance system 100 using an autonomously driven vehicle (hereinafter referred to as an evacuation guidance system) according to the present invention. The evacuation guidance operation system 100 comprises an autonomous vehicle 20, an operation server 30, a building 40, facility equipment including an elevator installed in the building 40, a fire alarm system installed in the building 40, and a network 50. The facility equipment including the elevator may be opening and closing devices other than the elevator, such as security doors, gates, automatic doors, and shutters. The fire alarm system is connected to the operation server via the network. In this invention, the terms "roads," "surrounding environment," and "facility equipment including elevators" refer to facility equipment used during evacuation guidance in the event of a fire or the like, and the terms "roads" and "surrounding environment" include the area around the first floor of the building that serves as an exit to a general road such as a public road.

[0015] The network 50 may have any configuration, and may be a dedicated line network, a public line network, a wireless network, or a wireless LAN such as Wi-Fi, etc. The network 50 may also be a network in which these networks are mixed.

[0016] As shown in Fig. 2, autonomous vehicle 20 may be composed of a main body 21 formed in a substantially cubic shape in appearance, and wheels 22 serving as a running unit suspended from main body 21. Main body 21 of autonomous vehicle 20 is equipped with a display unit 23, a surveillance camera 24, a speaker 25 serving as a first audio output unit, a luggage storage unit 26, lights 27, and turn signals 28, and as lights 27, front lights 27a and 27b are provided at the front of autonomous vehicle 20, and back lights 27c and 27d are provided at the rear. In the illustrated example, an autonomous vehicle equipped with luggage storage unit 26 for delivering various goods such as food and beverages and merchandise is shown, but if autonomous vehicle 20 is a vehicle for various tasks such as cleaning, disinfection, and monitoring, it may be equipped with various functions depending on the application, such as a polisher, cleaner, disinfectant sprayer, and alarm. When the autonomous vehicle 20 is a cart type equipped with a single seat, it can be used as an autonomous passenger vehicle to transport users such as the elderly, the sick, or the physically disabled. In the following, the autonomous vehicle 20 will be described as a merchandise delivery vehicle that travels along a route that includes sidewalks, corridors inside a building 40, elevators (described later), and the like.

[0017] The turn indicators 28 are arranged in an ear-like shape on the upper left and right sides of the front of the autonomously driven vehicle 20 so that people around the autonomously driven vehicle 20 can see the direction to turn from the front, sides, and rear. They are composed of, for example, light-emitting diode (LED) lamps. The display unit 23, serving as a first image output unit, is provided on the front side of the main body 21 and displays an eye-shaped image, which will be described later. The display unit 23 is composed of a display 63, which will be described later, and a pair of left and right openings 23a, 23b provided on the front side of the main body 21. The display 63 is preferably a display device using liquid crystal or light-emitting diodes, and more preferably a full-color light-emitting diode display with high-brightness light-emitting diodes arranged at a high density. A speaker 25 is provided below the display unit 23.

[0018] The surveillance cameras 24 are cameras that capture images of the surroundings of the autonomous vehicle 20, i.e., cameras that capture and monitor road conditions, pedestrians, and the like in front of, to the sides of, and behind the autonomous vehicle 20. The surveillance cameras 24 are composed of a pair of front surveillance cameras 24a, 24b provided on the left and right sides of the front of the main body 21, and a pair of side surveillance cameras 24c, 24d provided on the left and right sides of the main body 21. The surveillance cameras 24 may be monocular cameras, wide-area cameras, stereo cameras, and the like. As shown in FIG. 2, the surveillance cameras 24 may also be provided with a pair of left and right rear surveillance cameras 24e, 24f at the rear of the main body 21. If the front surveillance cameras 24a, 24b and the rear surveillance cameras 24e, 24f are each monocular cameras, images of pedestrians, bicycles, motorbikes, people in cars, and the like within a 360-degree area around the autonomous vehicle 20 can be captured, enabling the surroundings to be monitored. As shown in FIG. 2(B), a storage section 26 is provided on the rear side of the main body section 21 to accommodate cargo to be delivered.

[0019] 3 shows the configuration of a control system for autonomous vehicle 20. Autonomous vehicle 20 includes a CPU 61 as a control unit 29, which is connected to a speaker 25 as a first audio output unit, a battery 62, a display 63, a storage unit 64, a drive unit 65, and a detection unit 66. The detection unit 66 is made up of a monitoring camera 24 as an imaging unit, an attitude detection sensor 66a such as an IMU that is a sensor for controlling the traveling state of autonomous vehicle 20, a monitoring sensor 66b, a distance sensor 66c, a position sensor 66d, a bumper sensor 66e that detects contact with surrounding pedestrians, bicycles, motorcycles, etc., and a status transmission / reception unit 66f.

[0020] The CPU 61 and the storage unit 64 control each part of the device mounted on the autonomous vehicle 20. The CPU 61 may be configured with a microprocessor, a microcontroller, or the like. The storage unit 64 is configured with a non-volatile memory such as a DRAM, a hard disk drive (HDD), or a flash memory. The CPU 61 and each part of the device may be connected using a well-known method, for example, a controller area network (CAN). A program that causes the autonomous vehicle 20 to function is recorded and stored in a storage medium readable by a computer that executes the program and includes the CPU 61 and the storage unit 64. The storage medium may be a CD-ROM, a DVD-ROM, a USB memory, or the like. The program may be downloaded from the operation server 30 to the storage unit 64 of the computer via the network 50. The battery 62 is a power source for the autonomous vehicle 20. The battery 62 is connected to the CPU 61, and the remaining charge and other information of the battery 62 are monitored by the CPU 61.

[0021] The display 63 is a device that constitutes the display unit 23. In this embodiment, a predetermined eye shape image is displayed on the display unit 23 based on the state of the autonomous vehicle 20 and / or an image captured by the monitoring camera 24. Here, the state of the autonomous vehicle 20 refers to each driving operation state on the driving path, such as going straight, turning left, turning right, or stopping, as well as image information from the monitoring camera 24 and detection information from the detection unit 66 regarding the autonomous vehicle 20. The detection information from the detection unit 66 includes information from the attitude detection sensor 66a, the monitoring sensor 66b, the distance sensor 66c, the position sensor 66d, the bumper sensor 66e, etc. The image information from the monitoring camera 24 and the detection information from the detection unit 66 regarding the state of the autonomous vehicle 20 are transmitted to the operation server 30 (described later) by the status transmission / reception unit 66f as needed. An image according to a predetermined eye shape pattern is displayed on the display 63 in response to a light emission command from the CPU 61 based on a detection signal from the monitoring camera 24. When an image is displayed on the display 63, a predetermined output linked to the display of the image from the speaker 25 may also be generated. Furthermore, lights 27 and turn indicators 28 are displayed depending on the traveling or stopped state of autonomously driven vehicle 20. Lights 27 and turn indicators 28 are displayed by a light-emitting driver (not shown) in response to a light-emitting command from CPU 61. Here, in the present invention, images include not only still images but also moving images and other images.

[0022] Drive unit 65 is composed of motor 65a and driver 65b that drives and controls motor 65a, and motor 65a drives the aforementioned wheels 22. Specifically, driver L65b1 and driver R65b2 control motors 65a1 and 65a2, respectively, in response to control signals from CPU 61 to drive left rear wheel 22c and right rear wheel 22d. Speaker 25 is installed on the front of main body 21 of autonomously driven vehicle 20, as described above, and outputs a predetermined sound.

[0023] The attitude detection sensor 66a is, for example, an inertial measurement unit, which is an inertial measurement device that measures angular velocity and angular acceleration regarding movement around the roll axis, pitch axis, and yaw axis of the autonomously driven vehicle 20. The monitoring sensor 66b is a sensor that detects pedestrians, bicycles, motorbikes, cars, and obstacles around the autonomously driven vehicle 20, particularly in front of the vehicle, and is used to measure the distance to people and obstacles, i.e., distance measurement, perform two-dimensional and three-dimensional image recognition of people and obstacles, recognize shapes and colors, and follow the driving path of the autonomously driven vehicle 20.

[0024] The monitoring sensor 66b acquires position information, such as the position coordinates and the eye height of a person or obstacle, when it detects it. The display 63 may be controlled based on this position information. The monitoring sensor 66b may be configured with a module that acquires position information using a monocular camera or a stereo camera, a LIDAR, or the like. The module may be configured with a CPU or GPU (image processing unit) that processes image data acquired from the monocular camera or stereo camera to generate position information, a storage device, and the like. The module using the monocular camera can recognize the shape, color, and pattern of people and obstacles and can also measure approximate distances. The module using the stereo camera is used for ranging, three-dimensional recognition of people, vehicles, obstacles, etc., and identifying their shapes and colors. The LIDAR (Laser Imaging Detection and Ranging) performs laser image detection and ranging. The LIDAR may be a two-dimensional LIDAR or a three-dimensional LIDAR. The three-dimensional LIDAR can detect laser images ahead of the autonomous vehicle 20, measure the distance to the detected object, and measure the shape of the detected object. If the autonomous vehicle 20 is equipped with a lidar, it detects an object ahead of the autonomous vehicle 20 and the distance thereto, and sends data on the distance between the laser image ahead and the detected object to the CPU 61 as a detection signal.

[0025] The monitoring sensor 66b may be disposed on the upper front portion of the autonomous vehicle 20. The monitoring sensor 66b can detect images and measure distances to pedestrians, bicycles, motorcycles, cars, and other objects ahead of the autonomous vehicle 20 over long distances. The distance sensor 66c is a sensor that measures the distance between the autonomous vehicle 20 and obstacles. It is disposed facing the front of the autonomous vehicle 20 and measures the distance to an obstacle ahead on the road by emitting ultrasonic or infrared waves and detecting the reflected waves. The position sensor 66d acquires the current position of the autonomous vehicle 20. In this embodiment, a GNSS receiving unit is used. The position sensor 66d and the attitude detection sensor 66a may be separate devices, or may be a device that integrates GNSS receiving, gyro sensor, and acceleration sensor functions into a single package. The bumper sensor 66e can detect contact with nearby pedestrians, bicycles, motorcycles, and other objects and can stop the autonomous driving of the autonomous vehicle 20 or initiate an emergency stop. The status transmitting / receiving unit 66f is configured from a communication module capable of public communication, such as third generation (referred to as 3G), fourth generation (referred to as 4G), fifth generation (referred to as 5G), or wireless LAN.

[0026] Here, operation server 30 can remotely monitor and / or remotely operate the operation of autonomous vehicle 20 based on the current position and driving route 61c of autonomous vehicle 20, the relative positions of autonomous vehicles 20 with each other, or between autonomous vehicle 20 and the surrounding environment, etc., and elevator control unit 44 may be controlled by this operation server. In this case, autonomous vehicle 20 is remotely monitored and / or remotely operated by operation server 30, and when autonomous vehicle 20 approaches elevator 41 of building 40, operation server 30 controls elevator control unit 44 of building 40, allowing autonomous vehicle 20 to smoothly use elevator 41 of building 40.

[0027] In the control circuit of autonomously driven vehicle 20 configured as described above, CPU 61 controls drive unit 65 to drive autonomously vehicle 20 along a previously created travel route based on detection signals from sensors including attitude detection sensor 66a, monitoring sensor 66b, distance sensor 66c, and position sensor 66d, as well as monitoring camera 24. Map data 31 for creating the travel route is registered in operation server 30, and CPU 61 transmits current position information 61a of autonomously driven vehicle 20 and input destination information 61b to operation server 30, and creates travel route 61c based on map data 31 sent from operation server 30. If travel route 61c includes elevator 41 in building 40, CPU 61 creates elevator usage information 61d and elevator control information 61e and sends them to operation server 30 via network 50. The elevator usage information 61d is information regarding the elevator 41 in the building 40, including the boarding floor and the disembarking floor, and the elevator control information 61e is information including the call of the elevator car 43 and the designation of the disembarking floor after boarding. The CPU 61 also transmits the current location and running status to the operation server 30 via the network 50. This enables the operation server 30 to recognize the current location and running status of the autonomously driven vehicle 20 and manage its operation.

[0028] In this embodiment, autonomous vehicle 20 is an electric vehicle that uses battery 62 as a power source and travels by driving wheels 22 with motors 65a of drive unit 65. Autonomous vehicle 20 displays a predetermined eye shape image on display unit 23 based on its traveling state and images captured by monitoring camera 24 and / or monitoring sensor 66b. Furthermore, autonomous vehicle 20 is provided with speaker 25 as a first audio output unit on the front surface of main body 21, and outputs a predetermined audio to speaker 25 based on the traveling state of autonomous vehicle 20 and / or images captured by monitoring camera 24.

[0029] (Operation server) The operation server 30 constitutes a so-called robot management platform and is installed at an appropriate location. The operation server 30 is composed of a memory unit 32 that registers map data 31 necessary for creating a driving route for the autonomously driven vehicle 20, a control unit 33 that reads and writes the map data 31 from and to the memory unit 32 and manages the operation of the autonomously driven vehicle 20, an analysis unit 34 that analyzes the driving of the autonomously driven vehicle 20 and the use of elevators, a driving management unit 36 ​​that creates the driving route for the autonomously driven vehicle 20 and manages the driving of the autonomously driven vehicle 20, a collaboration unit 37, and a remote monitoring unit and / or remote operation unit 38. The operation server 30 is composed of a CPU, a GPU, a communication chip, DRAM, and non-volatile memory such as an HDD or SSD.

[0030] Here, the map data 31 is three-dimensional data that includes data on the roads and surrounding environment necessary for the travel of the autonomous vehicles 20, such as road boundaries and traffic light locations, for areas in which the autonomous vehicles 20 can travel, as well as facility equipment such as rooms on each floor of the multiple buildings 40, corridors serving as passageways, and security gates including elevators.

[0031] The memory unit 32 stores map data 31 as a database, and can read out map data 31 including the vicinity of location information based on the location information. Furthermore, in the case of a building 40 equipped with an elevator 41, the map data 31 includes information such as the location of the elevator 41 that can be used to travel to other floors within the building 40.

[0032] Control unit 33 of operation server 30 reads map data 31 of the area corresponding to a travel route 61c from the current position to the destination from storage unit 32 based on current position information 61a and destination information 61b sent from autonomously driven vehicle 20, and transmits the map data 31 to autonomously driven vehicle 20 via network 50. Furthermore, control unit 33 of operation server 30 transmits elevator usage information 61d sent from autonomously driven vehicle 20 to management server 45 of building 40 via network 50.

[0033] The analysis unit 34 of the operations server 30 has a function of analyzing elevator usage statistical information 61f sent from the elevator control unit 44, image data from surveillance cameras, etc. Specifically, when the autonomously driven vehicle 20 is a van-type automated delivery vehicle that transports goods, the analysis unit 34 calculates at-home probability information 61g for each floor obtained by analyzing the elevator usage statistical information 61f. Based on this at-home probability information 61g, the elevator control unit 44 sends the elevator usage statistical information 61f to the operations server 30 via the gateway 45a and the management server 45.

[0034] The driving management unit 36 ​​of the operation server 30 has a function for creating driving routes for multiple autonomous vehicles 20, a function for managing multiple autonomous vehicles 20, a task management function for assigning tasks to multiple autonomous vehicles 20 and creating driving schedules, and a function for setting up linkage with facility equipment such as security doors including elevators.

[0035] The driving route creation function of the driving management unit 36 ​​generates driving routes, sets priority areas, no-driving areas, evacuation routes, and the like for multiple autonomous vehicles 20. If the multiple autonomous vehicles 20 are manufactured by different companies, the management function of the driving management unit 36 ​​can manage them in groups by manufacturer. Data related to driving management used by the driving management unit 36 ​​is stored in a driving management database 32c in the storage unit 32. The driving management unit 36 ​​can collectively manage multiple autonomous vehicles 20. In the operation server 30 of the autonomous vehicle operation system 10B, the multiple autonomous vehicles 20 are remotely monitored and / or remotely operated by a remote monitoring unit and / or remote operation unit 38.

[0036] The coordination unit 37 of the operation server 30 has the function of communicatively connecting the multiple autonomous vehicles 20, the multiple elevators 42B installed in each building, and security gates and the like as necessary. The coordination unit 37 supports various communication protocols to accommodate communication methods specific to the manufacturers of the multiple autonomous vehicles 20. Examples of communication methods include MQTT (MQ Telemetry Transport, a protocol used for communication between devices), PLC (Programmable Logic Controller), and OPC UA (an open source industrial interface established by the OPC Foundation). Data corresponding to these communication methods used by the coordination unit 37 is stored as a connection database 32a in the storage unit 32.

[0037] The linking unit 37 also connects a plurality of elevators 42B arranged in each building with security gates and the like using an API (Application Program Interface) stored in the connection database 32a.

[0038] The remote monitoring unit and / or remote control unit 38 has the function of remotely monitoring and remotely controlling the multiple autonomous vehicles 20 located in each building. The control unit 33B registers the multiple autonomous vehicles 20 to be monitored in the driving management unit 36. The status of the registered multiple autonomous vehicles 20 is periodically monitored by the remote monitoring unit and / or remote control unit 38. Image information from the monitoring camera 24 of each autonomous vehicle 20 and detection information from the detection unit 66 are transmitted to the remote monitoring unit and / or remote control unit 38 by the status transmission / reception unit 66f as needed. For example, current location information 61a from each autonomous vehicle 20i is transmitted from the control unit 33B to the remote monitoring unit and / or remote control unit 38 via the network. Data related to remote monitoring and / or remote control used by the remote monitoring unit and / or remote control unit 38 is stored in a remote monitoring and / or remote control database 32d in the storage unit 32.

[0039] The current position information 61a of each automatically driven vehicle 20a is displayed on a map, thereby displaying the current position information 61a of multiple automatically driven vehicles 20. The current position information 61a may be displayed on a display of the operation server 30. The current position information 61a may be updated continuously or at predetermined time intervals and displayed on the display. In this way, the current position information 61a of multiple automatically driven vehicles 20 is displayed on the map in real time.

[0040] When multiple autonomous vehicles 20 initially connect to the operation server 30 via the network 50, the control unit 33 of the operation server 30 controls the linking unit 27 to initiate communication using the communication method of each autonomous vehicle 20. The identification number (ID) of each autonomous vehicle 20 is detected, thereby recognizing the type, company, etc. of each autonomous vehicle 20. For each autonomous vehicle 20 recognized by the control unit, the driving management unit 36 ​​(described below) manages its tasks, such as work and work time. The analysis unit 34 has the function of analyzing data acquired from multiple autonomous vehicles 20 in addition to data acquired from facilities such as elevators, and the function of simulating various usage situations and usage of the autonomous vehicle operation system. Data for analysis used by the analysis unit 34B is stored in the analysis database 32b of the memory unit 32. Efficiency improvements and problems can be visualized based on the data acquired from multiple autonomous vehicles 20 and facilities such as elevators.

[0041] (Buildings and facility equipment, including elevators) Building 40 is made up of multiple floors, and to enable easy movement between floors, it is equipped with facility equipment including elevator 41, such as opening and closing devices (also called electrical equipment), such as security doors, gates (also called security gates), automatic doors, and shutters. Elevator 41 is made up of an elevator car 43 that is supported so as to be movable up and down within a travel path 42 that passes vertically through building 40, and is moved up and down by a drive unit (not shown), and an elevator control unit 44 that drives and controls elevator car 43 up and down. Elevator control unit 44 is connected to a management server 45 that manages the entire building 40 via gateway 45a, and management server 45 is further connected to operation server 30 from gateway 45b via network 50. As a result, in the operation system for autonomous vehicle 20, autonomous vehicle 20, operation server 30, and management server 45 are interconnected with building 40 and the facility equipment control unit including elevator 41, and the facility equipment including elevator 41 in building 40 can be controlled by control unit 33 of operation server 30 via network 50. In the following explanation, the control unit of the facility equipment, including opening and closing devices such as security doors, gates, automatic doors, and shutters, including elevators used or traveled by autonomous vehicle 20, will also be simply referred to as the facility equipment control unit.

[0042] A speaker 43a serving as a second audio output unit is provided inside the elevator car 43, and a speaker 41a serving as a third audio output unit is provided in the elevator hall of each floor of the building 40. A display 43b serving as a second image output unit may also be provided inside the elevator car 43. Furthermore, a camera 43c for monitoring the passenger status inside the elevator car 43 may be provided above or on the ceiling inside the elevator car 43. An image signal acquired by the camera 43c is sent from the elevator control unit 44 to the management server 45 via the gateway 45a, and is also sent from the management server 45 to the operation server 30 via the gateway 45b and the network 50.

[0043] As a result, the elevator control unit 44 sends a signal to call the elevator when the autonomous vehicle 20 approaches the elevator hall based on the current position 61a of the autonomous vehicle 20, which is sent from the autonomous vehicle 20 via the network 50 through the operation server 30 and management server 45, and when the autonomous vehicle 20 gets into the elevator car 43, it sends an input signal for the destination floor, allowing the user to use the elevator 41.

[0044] Furthermore, the control unit 44 of the facility equipment other than the elevators can use automatic doors by sending a signal to open an automatic door when the self-driving vehicle 20 approaches an automatic door or the like, and a signal to close the automatic door after the self-driving vehicle 20 has passed, based on the current position 61a of the self-driving vehicle 20 sent from the self-driving vehicle 20 via the network 50, through the operation server 30 and the management server 45. Security doors, gates (also called security gates), automatic doors, and shutters other than automatic doors are also controlled by the facility equipment control unit 46 in the same way.

[0045] When autonomous vehicle 20 uses elevator 41 in building 40, control unit 61 of autonomous vehicle 20 transmits elevator usage information 61d and elevator control information 61e to elevator control unit 44 of building 40 via network 50. As a result, when autonomous vehicle 20 gets into elevator car 43, second audio output unit 43a in elevator car 43 outputs an audio announcement regarding the use of elevator 41 of autonomous vehicle 20. Along with the announcement regarding the use of elevator 41 from second audio output unit 43a, second image output unit 43b in elevator car 43 may output an image regarding the use of elevator 41 of autonomous vehicle 20 based on elevator usage information 61d.

[0046] Even if a person in elevator car 43 misses or has difficulty hearing the announcement output by voice from first audio output unit 25 of autonomous vehicle 20, they can be sure to recognize that elevator 41 of autonomous vehicle 20 will be used by listening to the announcement output by voice from second audio output unit 43a provided in elevator car 43. Furthermore, even if they have difficulty hearing due to hearing loss or are hearing impaired, they can be sure to recognize that elevator 41 of autonomous vehicle 20 will be used by viewing image information related to the use of elevator 41 of autonomous vehicle 20 on display 43b in elevator car 43.

[0047] (Elevator hall) FIG. 4 is a schematic diagram showing the configuration of an elevator hall 46. The elevator hall 46 is equipped with elevator doors 46a, a display unit 46b indicating elevator movement, push buttons 46c indicating up and down for desired destinations, a speaker (third audio output unit) 41a, and a display 46e as a third image output unit. The image and audio output of the speaker 41a and the display 46e are controlled by the operation server 30 and / or the elevator control unit 44 based on elevator usage information 61d. Audio output by the speaker 41a can be performed in the same manner as in the flow diagram shown in FIG. 6, which will be described later. In addition to the audio output by the speaker 41a, images related to boarding and disembarking the autonomously driven vehicle 20 are displayed on the display 46e. These images can be still images or animated images of the autonomously driven vehicle 20, or images of the autonomously driven vehicle 20 captured by a camera installed in the elevator car 43. The location where the speaker 41a and the display 46e are installed is not limited to the elevator hall 46, but may be in the elevator car 43 or in the vicinity of the elevator hall 46, such as a location close to the elevator hall 46.

[0048] (Use of cameras installed near facility equipment, including elevators) To monitor the movements of autonomous vehicles 20 and people within building 40, cameras may be installed near facility equipment, including elevator halls 46, elevator cars 43, security doors, gates, automatic doors, shutters, and other opening and closing devices. Image signals acquired from these cameras are sent to control unit 33 of operation server 30 and / or the management server.

[0049] (Using cameras installed in elevator cars) If the camera 43c is installed inside the elevator car 43, for example, on the ceiling, the elevator control unit 44 or the management server 45 calculates the number of passengers inside the elevator car 43 from the image signal acquired by the camera 43c. Various image recognition methods can be used to calculate the number of passengers. For passengers getting on and off the autonomous vehicle 20, the ratio of the number of passengers to the passenger capacity of the elevator car 43, i.e., the occupancy rate, may be calculated. The passenger information 43e of the elevator car 43, including the number of passengers and the occupancy rate acquired by the elevator control unit 44 or the management server 45, may be sent to the control unit 33 of the operation server 30.

[0050] The camera 43c may be a stereo camera capable of measuring distance, a TOF camera (Time of Flight Camera), LiDAR, or the like. An integrated sensor may be used to calculate the number of passengers and their luggage in the elevator car 43 from image signals acquired by the camera 43c without using the elevator control unit 44 or management server 45. This sensor is called an elevator analysis sensor because it acquires analytical data on the number of passengers and occupancy rate in the elevator car 43 based on images and / or distance information from the camera 43c, TOF camera, LiDAR, or the like. The elevator analysis sensor may be configured as a modular sensor, for example, including an image sensor serving as a stereo camera or a TOF image sensor, a computer, a storage device, a communication board, and the like. The elevator analysis sensor is installed on the ceiling of the elevator car 43. Occupant information 43e, which is analysis data such as the number of people in the elevator and the number of autonomously driven vehicles 20 from the elevator analysis sensor, may be sent to the control unit 33 of the operation server 30 via the elevator control unit 44 or management server 45 and network 50. The occupant information 43e from the elevator analysis sensor is analysis data related to the number of users in the elevator car 43 and the number of autonomously driven vehicles 20, and does not include image data, so there is no leakage of privacy.

[0051] The occupant information 43e is sent from the control unit 33 to the CPU 61 of the autonomous vehicle 20 via the network 50. At this time, if multiple elevators 41 are installed in the building 40, the CPU 61 references the occupant information 43e to create usage information 61d and elevator control information 61e to advise the use of an available elevator, and transmits these to the operation server 30 via the network 50. The usage information 61d and elevator control information 61e created based on the occupant information 43e obtained by the camera 43c or elevator analysis sensor installed in the elevator car 43 allow the autonomous vehicle 20 to move to and board an elevator 41 that has space available for the autonomous vehicle. In this way, the autonomous vehicle 20 can refer to the occupant information 43e in advance to detect elevators that are full of users or elevators that are full of users and other autonomous vehicles, and can easily select an available elevator.

[0052] (Fire alarm equipment) As shown in FIG. 1 , the fire alarm system 52 includes a receiver 53, which is installed in a caretaker's room or the like of the structure 40. A first transmission path 54a of the receiver 53 is connected to a fire detector 55 and a transmitter 56 that are located in a monitored area within the structure. When a fire occurs, a person near the fire site presses a push button switch on the transmitter 56, which transmits a fire notification signal to the receiver 53. Furthermore, a second transmission path 54b for controlling the receiver 53 is connected to terminal devices such as a district sound device 57, a fire door 58, and a fire shutter 59. The fire notification signal transmitted to the receiver 53 is reported to the management server 45, and is also reported from the management server 45 to the operation server 30.

[0053] Next, normal operation of autonomously driven vehicle 20 according to this embodiment will be described with reference to Fig. 5. Here, the autonomous driving of autonomously driven vehicle 20 when elevator 41 of building 40 is not used during driving route 61c will be described. First, in step A1, destination information 61b indicating a destination is input to the CPU 61 of automatically driven vehicle 20 via an operation unit (not shown) or from the operation server 30. Next, in step A2, the CPU 61 of automatically driven vehicle 20 detects the current position of the automatically driven vehicle 20 based on a detection signal from the position sensor 66d and acquires current position information 61a. In step A3, the CPU 61 transmits the current position information 61a and the destination information 61b to the operation server 30 via the network 50. In response to this, in step A4, the control unit 33 of the operation server 30 reads, from the storage unit 32, map data 31 of the area corresponding to the driving route from the current position indicated by the current position information 61a to the destination indicated by the destination information 61b based on the current position information 61a and the destination information 61b, and in step A5, transmits this map data 31 to the automatically driven vehicle 20 via the network 50.

[0054] In step A6, CPU 61 creates a driving route 61c from the current location to the destination based on current location information 61a, destination information 61b, and map data 31, and transmits driving route 61c to operation server 30 via network 50. In step A7, CPU 61 drives and controls drive unit 65 based on driving route 61c to cause automatically driven vehicle 20 to automatically drive along driving route 61c. During automatic driving, CPU 61 of automatically driven vehicle 20 appropriately outputs audio from speaker 25 to notify surrounding parties of the presence and driving of automatically driven vehicle 20. In step A8, if CPU 61 detects an object such as a pedestrian, bicycle, motorcycle, other vehicle, or obstacle based on detection signals from detection unit 66, i.e., attitude detection sensor 66a, monitoring sensor 66b, distance sensor 66c, and position sensor 66d, and monitoring camera 24, CPU 61 changes driving route 61c to avoid the object, and drives and controls drive unit 65 to stop or make an emergency stop. Operation server 30 monitors the autonomous driving of autonomous vehicle 20 based on driving route 61c received from autonomous vehicle 20 before autonomous driving begins and current location information 61a successively sent from autonomous vehicle 20, and may be configured to stop or make an emergency stop for autonomous vehicle 20 if autonomous vehicle 20 significantly deviates from driving route 61c. When autonomous vehicle 20 reaches the destination in step A10, CPU 61 controls the drive of drive unit 65 in step A11 to stop the autonomous driving, and autonomous driving ends.

[0055] The automatic travel of automatically driven vehicle 20 when elevator 41 in building 40 is used will be described in further detail below with reference to the flowchart in FIG. After creating the travel route 61c in the aforementioned step A6, the CPU 61 creates elevator usage information 61d and elevator control information 61e in step B1, and transmits the elevator usage information 61d and elevator control information 61e to the operation server 30 via the network 50 in step B2.

[0056] In step B3, when autonomous vehicle 20 approaches the elevator hall of elevator 41 in building 40, in step B4, operation server 30 detects that autonomous vehicle 20 is approaching the elevator based on current location information 61a from autonomous vehicle 20, and in step B5, operation server 30 transmits elevator usage information 61d and elevator control information 61e from management server 45 to elevator control unit 44 via network 50. In response, elevator control unit 44 calls elevator 41 at the relevant floor based on elevator control information 61e of elevator usage information 61d and elevator control information 61e in step B6, and also outputs a voice message from speaker 43a in elevator car 43 as a second voice output unit based on elevator usage information 61d saying, "An autonomous vehicle will board from floor ○ to floor □. Please cooperate with boarding," to provide advance notice to passengers in elevator car 43 that autonomous vehicle 20 will board. If the elevator car 43 is equipped with a second image output unit 43b, in step B7', the display 43b in the elevator car 43 may display an image, "An autonomous vehicle will board from floor ○ to floor □. Please cooperate with boarding." along with the audio output of step B7, based on the elevator usage information 61d.

[0057] Simultaneously with step B7, in step B8, elevator control unit 44 outputs a voice message from speaker 41a provided in the elevator hall of the relevant boarding floor as a third voice output unit, saying, "An autonomously driven vehicle will board from floor ○ to floor □. Please cooperate with boarding," to inform people near the elevator hall of the relevant boarding floor that autonomously driven vehicle 20 will board. Furthermore, if elevator hall 46 is equipped with third image output unit 46e, in step B8', similar text may be displayed as an image on display 46e in elevator hall 46 based on elevator usage information 61d, along with the above voice output.

[0058] When elevator car 43 arrives at the boarding floor and the doors open in step B9, elevator control unit 44 outputs a voice message from speaker 43a inside elevator car 43 saying, "You are at floor X. An autonomously driven vehicle is boarding now. Please help board," in step B10 to guide people inside elevator car 43 to board autonomously driven vehicle 20. If elevator car 43 is equipped with second image output unit 43b, in step B10', display 43b inside elevator car 43 may similarly display an image along with the voice output in step B10 based on elevator usage information 61d.

[0059] Simultaneously with step B10, in step B11, elevator control unit 44 outputs a voice message from speaker 41a provided in the elevator hall of the relevant floor saying, "An autonomously driven vehicle is about to board. Please cooperate with boarding," to inform people near the elevator hall of the relevant floor that autonomously driven vehicle 20 is boarding. If elevator hall 46 is equipped with third image output unit 46e, in step B11', a similar message may be displayed as an image on display 46e in elevator hall 46 along with the above voice message, based on elevator usage information 61d.

[0060] When autonomous vehicle 20 enters elevator car 43 and the doors of elevator car 43 close in step B12, elevator control unit 44 specifies the disembarking floor for elevator car 43 based on elevator control information 61e in step B13, and outputs a voice message from speaker 43a in elevator car 43 based on elevator usage information 61d in step B14 saying, "An autonomous vehicle has boarded at floor ○. This autonomous vehicle will disembark at floor □. Please cooperate with disembarking," to inform passengers in elevator car 43 that autonomous vehicle 20 will disembark. If elevator car 43 is equipped with second image output unit 43b, similar text is displayed on display 43b in elevator car 43 based on elevator usage information 61d in step B14'.

[0061] In step B15, when elevator car 43 starts ascending or descending from the boarding floor, in step B16, elevator control unit 44, based on elevator usage information 61d, outputs a voice message from speaker 41a provided in the elevator hall at the disembarking floor of automatically driven vehicle 20 saying, "The automatically driven vehicle will now disembark. Please cooperate with disembarking," to provide advance notice of automatically driven vehicle 20 disembarking. If elevator hall 46 is equipped with third image output unit 46e, in step B16', a similar image is displayed on display 46e in elevator hall 46 together with the above voice message based on elevator usage information 61d.

[0062] When elevator car 43 arrives at the disembarking floor and the doors open in step B17, elevator control unit 44 outputs a voice message from speaker 43a inside elevator car 43 saying, "This is floor X. The autonomously driven vehicle is disembarking. Please cooperate with disembarking," in step B18, informing people inside elevator car 43 that they will be disembarking autonomously driven vehicle 20. If elevator car 43 is equipped with second image output unit 43b, in step B18', display 43b inside elevator car 43 similarly displays an image together with the voice output in step B12.

[0063] Simultaneously with step B18, in step B19, elevator control unit 44 outputs a voice message from speaker 41a installed in the elevator hall of the relevant floor, saying, "The autonomously driven vehicle will now disembark. Please cooperate with disembarking," to inform people near the elevator hall of the relevant floor that autonomously driven vehicle 20 is disembarking. If elevator hall 46 is equipped with a third image output unit 46e, in step B18', a similar image is displayed on display 46e in elevator hall 46 along with the above voice output. As a result, in step B20, autonomous vehicle 20 disembarks from elevator car 43. In this way, autonomous vehicle 20's use of elevator 41 installed in building 40 is completed.

[0064] In the above description, when using elevator 41, autonomous vehicle 20 does not output audio from speaker 25. However, it may also be possible to provide audio guidance regarding elevator use of autonomous vehicle 20 as appropriate using speaker 43a, which is the second audio output unit, and speaker 41a, which is the third audio output unit. Furthermore, elevator control unit 44 may output an audio announcement from second audio output unit 43a, based on elevator usage information 61d, from before boarding the elevator of autonomous vehicle 20 to before disembarking, informing passengers that they will be boarding an autonomous vehicle from the boarding floor to the disembarking floor. By listening to the announcement from second audio output unit 43a informing passengers that they will be boarding an autonomous vehicle, passengers in elevator 41 can understand that they will be using the elevator of autonomous vehicle 20 before boarding the autonomous vehicle 20. Elevator control unit 44 may be provided with a third audio output unit 41a in the elevator hall of each floor of elevator 41, and elevator control unit 44 may output an audio announcement regarding elevator use of autonomously driven vehicle 20 from third audio output unit 41a provided at the boarding floor or disembarking floor based on elevator use information 61d. With this configuration, people in the elevator hall to use elevator 41 can listen to the announcement regarding elevator use of autonomously driven vehicle 20 from third audio output unit 41a and take action such as giving way to other people when boarding autonomously driven vehicle 20 or moving out of the way to allow autonomously driven vehicle 20 to disembark when disembarking autonomously driven vehicle 20.

[0065] (Evacuation Guidance System) The evacuation guidance system of the present invention will be described assuming that the autonomous vehicle is a delivery robot. The fire alarm system outputs a fire alarm when a receiver receives a fire notification signal from a fire detector installed in the building, and outputs a fire information signal including the location of the fire to the operation server 30 and also to the outside. Specifically, the fire notification signal is transmitted from the management server 45 to the control unit 33 of the operation server 30 via the gateway and network 50. The outside refers to the manager's room or the management server 45.

[0066] When a fire information signal is issued, the fire information signal is reported to the elevator control unit 44, or is reported to the elevator control unit 44 via the management server 45, and the elevator control unit 44 switches to control of the fire outbreak mode. In this fire outbreak mode, the elevator control unit 44 and / or the management server 45 cuts off the connection between the elevator 41 and the operation server 30, and the elevator 41 switches to direct operation to the evacuation floor. Typically, the evacuation floor is the first floor, which is the entrance to the building 40. Furthermore, if an emergency elevator is installed in the building 40, the emergency elevator can also be used for evacuation in some cases.

[0067] In the fire outbreak mode, the management server 45 controls the elevator control unit 44 to switch to direct operation to the evacuation floor, and controls the control unit 48 of the facility equipment, including the elevator 41, to open the facility equipment, including opening and closing devices such as security doors, gates, automatic doors, and shutters, to ensure unhindered movement of people and autonomous vehicles 20 within the building 40. At this time, it is preferable for safety reasons that the management server 45 transmits elevator operation information for the fire outbreak mode within the building 40 to the operation server 30 along with a fire information signal. Furthermore, if a security door, gate, automatic door, shutter, or other opening and closing device fails to open due to a malfunction, the operation server 30 is notified of this as malfunction information for the automatic door, etc. Typically, facility equipment usage information 48a is sent from autonomous vehicle 20 to elevator control unit 44 and / or facility equipment control unit 48. In response, elevator control unit 44 and / or facility equipment control unit 48 transmits facility equipment control information 48b to facility equipment control unit 48, which includes the elevator. Plans and information regarding autonomous vehicle 20's use of the facilities within the building, i.e., facility equipment usage information 48a, may be transmitted from cooperation unit 37 of operation server 30 to facility equipment control unit 48 via network 50.

[0068] In response to this, the control unit 33 of the operation server 30 displays the location of the fire on the display using the fire notification signal, checks for self-driving vehicles 20 working within the building, and if necessary creates an evacuation route using available elevators 41 within the building, and extracts self-driving vehicles 20 that are not working using a pre-created evacuation procedure, and sends an evacuation route guidance instruction signal to this extracted self-driving vehicle 20 to guide it along the evacuation route.

[0069] When the operation server 30 receives a fire information signal from the receiver, it displays the fire location, elevator operation information, and malfunction information for automatic doors and other devices on the facility's map data, compares the information with pre-created evacuation routes, creates a safe evacuation route, and transmits a safety confirmation instruction signal to the autonomously driven vehicle 20 to issue a safety confirmation instruction signal at a predetermined safety confirmation position. When the control unit of the autonomously driven vehicle 20 receives a safety confirmation instruction signal from the operation server 30, the control unit 29 recognizes the current location based on the detection signal from the detection unit 66 and performs AI analysis on images of the surrounding environment acquired by the imaging unit, i.e., the monitoring sensor 66b, to determine whether the current location is safe and transmits the result to the operation server 30. Specifically, it acquires AI analysis information based on image signals including fire, smoke, and people acquired along the evacuation route, and outputs the AI ​​analysis information from the autonomously driven vehicle 20 to the control unit 48 of the facility equipment, including the elevator, and to the operation server 30. The imaging unit installed in the autonomously driven vehicle 20 may be an infrared camera. The AI ​​analysis may be performed by a computer that processes image signals acquired by an infrared camera and a storage device, etc., and may obtain AI analysis information based on image signals including fire, smoke, and people acquired along evacuation routes. If the current location is safe, the control unit receives map data of the area related to the evacuation route from the current location to the safety confirmation location from the operation server, creates a new evacuation route from the current location to the safety confirmation location, controls the drive unit 65 based on the evacuation route to drive the motor, and when moving through or using one or more facility equipment including the elevator 41, the control unit notifies nearby autonomous vehicles 20 and / or nearby people of the evacuation route.

[0070] FIG. 7 is a flowchart showing the procedure for guiding an autonomously driven vehicle 20 to evacuation in the autonomously driven vehicle evacuation guidance system of FIG. In step ST1, the autonomous vehicle 20 performs automatic delivery of goods within the building 40. In step ST2, the control unit of autonomous vehicle 20 checks whether or not a fire information signal has been received from operation server 30. If a fire information signal has been received (YES), in step ST3, the control unit of autonomous vehicle 20 stops delivery operations and waits at a predetermined location.

[0071] If the control unit of autonomously driven vehicle 20 receives a safety confirmation instruction signal from operation server 30 in step ST4 (YES), then in step ST5, it recognizes the current position based on the detection signal from the detection unit and performs AI analysis of the image of the surrounding environment acquired by the imaging unit, determines whether the current position is safe in step ST5, and transmits the result to operation server 30 in step ST6. Note that if the safety confirmation instruction signal is not received from operation server 30 (NO), then the autonomously driven vehicle waits at a predetermined location in step ST3. The operation server may use AI analysis to make decisions based on images acquired by the imaging unit 66b of the autonomous vehicle 20 or a surveillance camera installed in a building, including counting the number of passengers in elevators, adjusting evacuation routes, and resetting evacuation routes.

[0072] In step ST7, the control unit of autonomous vehicle 20 receives map data of the area related to the evacuation route from the current location to the safety confirmation position from operation server 30, creates a new evacuation route from the current location to the safety confirmation position, and in step ST8, controls the drive unit to drive the motor based on the evacuation route, and when moving along the evacuation route, passing through or using one or more facility equipment including elevators, the control unit outputs voice from the audio output while notifying nearby people of evacuation guidance, and the vehicle automatically travels to the safety confirmation position. The map data of the area related to the evacuation route received by the control unit of autonomous vehicle 20 in step ST7 includes elevator operation information and malfunction information for automatic doors, etc.

[0073] In step ST9, the control unit of autonomous vehicle 20 automatically travels to a safety confirmation position, and if evacuation guidance is completed (YES), the control unit returns to step ST1 and returns to delivery work for the building. Also, in step ST9, if autonomous vehicle 20 automatically travels to a safety confirmation position but evacuation guidance is not completed (NO), the control unit returns to step ST3 and waits at a predetermined location.

[0074] Evacuation guidance system 100 using autonomously driven vehicles according to the present invention includes management server 45 that controls building 40 as a whole, and management server 45 may be controlled by operation server 30 to control elevator control unit 44 based on elevator usage information 61d and elevator control information 61e. With this configuration, when autonomously driven vehicle 20 enters building 40, management server 45 recognizes autonomously driven vehicle 20 and can output an audio announcement regarding elevator use by autonomously driven vehicle 20 from second audio output unit 43a in elevator car 43, based on elevator usage information 61d and elevator control information 61e received directly from control unit 61 of autonomously driven vehicle 20 or via operation server 30. In other words, the control unit of the autonomous vehicle outputs an announcement from the audio output unit of the autonomous vehicle to notify people in the vicinity of the autonomous vehicle's evacuation guidance while the autonomous vehicle is traveling along an evacuation route, and also outputs an announcement about the use of facility facilities, including elevators, from the audio output unit of the autonomous vehicle based on usage information about the facility facilities, including elevators. Furthermore, if autonomous vehicle 20 is equipped with display unit 23, the evacuation guidance notification may be displayed on display unit 23 together with the audio output, based on information about evacuation guidance from operation server 30 and / or management server 45 of building 40. The display on display unit 23 may be performed together with the audio output.

[0075] (Evacuation guidance in the event of an earthquake) In the event of an earthquake, when the earthquake sensor detects shaking, the elevator 41 automatically stops at the nearest floor. After the elevator 41 arrives at the nearest floor, the doors automatically open. Therefore, when elevator stop information regarding the occurrence of an earthquake and elevator stop is sent to the operation server 30 via the elevator control unit 44 and / or management server 45, the operation server 30 notifies the autonomously driven vehicle 20 that was using the elevator 41 of the elevator stop information and instructs the vehicle 20 to exit the elevator 41 at the nearest floor where the elevator 41 will stop. Upon receiving this, the autonomously driven vehicle 20 exits the elevator 41 and, if there are people nearby, notifies them of the occurrence of an earthquake and that the elevator 41 will be stopped, provides evacuation guidance for the earthquake, and waits in a predetermined location, such as the elevator hall 46, until operation resumes.

[0076] Normally, in an earthquake with a seismic intensity of 5 or higher, operation of the elevator 41 is resumed after an inspection by the elevator 41's maintenance company. In addition, in an earthquake with a seismic intensity of 4 or lower, operation is resumed after a test run of one or more round trips and confirmation that the elevator 41 is operating normally. Therefore, when operation of the elevator 41 is resumed, elevator operation resume information is transmitted to the operation server 30 via the elevator control unit 44 and / or management server 45. In response to this, the operation server 30 transmits the elevator operation resume information to the waiting autonomous vehicle 20, instructing it to return to work.

[0077] According to the evacuation guidance system using an autonomous vehicle of the present invention, an evacuation route is created in advance by referring to map data, and in the event of a fire occurring within a building, the fire alarm equipment and the operation server 30 of the autonomous vehicle 20 work together to drive the autonomous vehicle 20 along the evacuation route, thereby guiding people within the building 40 to safety and moving it to a safety confirmation position. In addition, if an earthquake occurs in the building 40, the elevator control unit 44 and the operation server 30 will work together to drive the autonomous vehicle 20 along the evacuation route, guide people inside the building 40 to safety, and move it to a safety confirmation position.

[0078] (Second embodiment) As a second embodiment, an evacuation guidance system 100A will be described, which provides evacuation guidance using a voice guide (see Patent Document 5) and a display device using a portable terminal carried by a person inside a building 40. Figure 8 is a block diagram showing the configuration of the evacuation guidance system 100A when a person inside the building 40 is carrying a mobile terminal, Figure 9 is a workflow diagram for generating a data set in the operation server 30B, and Figure 10 is a workflow diagram for estimating a user's self-position in the operation server 30B.

[0079] As shown in Fig. 8, a mobile terminal 130 carried by a person inside a building 40 is connected to the operation server 30B via a wireless network 110 and a network 50. The wireless network 110 is a wireless network of any configuration to which the mobile terminal 130 can be connected, and may be a dedicated line network, a public line network used by mobile phones such as 4G or 5G, or a wireless LAN such as Wi-Fi.

[0080] The operation server 30B is installed at an appropriate location and is configured to include a memory unit 32 that registers map data (hereinafter referred to as map data) 31 for supporting pedestrians and others through audio guidance, and a control unit 33B that reads and writes the map data 31 from the memory unit 32.

[0081] Here, the map data 31 is three-dimensional data including data on roads and the surrounding environment necessary for providing audio guidance to pedestrians and the like in areas where pedestrians and the like can move, such as information on boundaries between sidewalks and roadways, steps, pedestrian signals, tactile paving blocks, buildings, stores, and the positions and heights of wireless LAN access points (described later). The height refers to the height position of the mobile terminal 130. The three-dimensional data of the map data 31 is preferably a three-dimensional point cloud map, which is a so-called high-resolution map.

[0082] Map data 31 is registered as a database in the storage unit 32, and map data including the vicinity of the location information can be read based on the location information. The map data preferably associates location information of a wireless LAN, for example, a building, a store, etc. where a WiFi (registered trademark) access point is installed with WiFi information regarding the SSID of the access point.

[0083] Furthermore, if the facility such as a building that the user is visiting spans multiple floors, the map data 31 may be created to include the number and location information of a specific elevator that the user can use to travel to other floors within the building 40.

[0084] As will be described later, the control unit 33B reads map data 31 of the area including the travel route from the current location to the destination sent from the mobile terminal 130 from the memory unit 32 and transmits the data to the mobile terminal 130.

[0085] The mobile terminal 130 is a mobile terminal with a known configuration, such as a smart device such as a smartphone or tablet, and is configured to be connectable to the wireless network 110. As shown in FIG. 1 , the mobile terminal 130 is configured with a transmitting / receiving unit 131, a control unit 132, a storage unit 133, a display unit 134, an imaging unit 135, a detection unit 136, an audio input unit 137, an audio output unit 138, and a vibrator 139.

[0086] The transmitting / receiving unit 131 connects to the wireless network 110 and transmits and receives data. The control unit 132 controls the transmitting / receiving unit 131, the storage unit 133, the display unit 134, the imaging unit 135, the detection unit 136, the audio input unit 137, the audio output unit 138, and the vibrator 139 to perform predetermined operations.

[0087] The display unit 134 is configured as a so-called touch panel, and is driven and controlled by the control unit 132 to display a predetermined image, and also allows predetermined input to be made by operating the displayed screen with a finger.

[0088] The imaging unit 135, which may be, for example, a still-image camera or a TOF (Time of Flight) camera, captures an image based on a predetermined operation, records the image signal 135a in the storage unit 133, and transmits it to the control unit 32. Images captured by the still-image camera or TOF camera may be still images or videos. A still-image camera calculates the distance to each pixel using a so-called triangulation method based on the parallax of the same object, thereby obtaining a three-dimensional image. A TOF (Time of Flight) camera is a camera equipped with a distance measurement sensor that uses distance measurement technology. The captured TOF image data contains distance information to the object for each pixel, which itself constitutes a three-dimensional image. For this reason, in recent years, smart devices such as smartphones have been equipped with TOF cameras instead of conventional cameras, or with TOF cameras in addition to conventional cameras.

[0089] The detection unit 136 includes an attitude detection sensor 136a, a position sensor 136b, and an ambient sensor 136d. It may further include a height sensor 136e. The attitude detection sensor 136a is composed of a so-called acceleration sensor and a gyro that detects angular velocity, and is also called an IMU (Inertial Measurement Unit). It may further include a magnetic sensor as needed. The three-dimensional position and direction of the mobile terminal 130, i.e., its attitude, is detected based on the acceleration and / or angular velocity of the mobile terminal 130 in the three-axis directions, and a detection signal S1 is output to the control unit 132.

[0090] The position sensor 136b is, for example, a GPS sensor, which detects the position of the mobile terminal 130, that is, the longitude and latitude, and outputs a detection signal S2 to the control unit 132.

[0091] The surrounding sensor 136c is provided on or outside the mobile terminal 130 and is configured with a LIDAR 136d. However, instead of a LIDAR, an external stereo camera or the like may be used. The LIDAR 136d is also called a laser radar and performs light detection and ranging (LIDAR) or laser imaging detection and ranging. A two-dimensional LIDAR or a three-dimensional LIDAR may be used as the LIDAR 136d. For example, a three-dimensional LIDAR can perform laser image detection of an object located in the vertical direction and measure the distance to the detected object. The LIDAR 136d improves the accuracy of distance measurement compared to conventional cameras and TOF cameras.

[0092] The surrounding sensor 136c detects objects around the mobile terminal 130, i.e., obstacles on the front, rear, left, and right sides, other pedestrians, cars, etc. When the surrounding sensor 136c detects an object on the movement path, it sends a detection signal S3 including the direction and distance of the object to the control unit 132.

[0093] According to the above configuration, by using not only the imaging signal 135a from the imaging unit 135 but also the detection signal S3 from the detection unit 136, i.e., the detection signal from the lidar 136e, the position and size of objects can be detected more accurately, and by correcting the travel route to avoid these objects, the user can travel along a safer travel route.

[0094] The voice input unit 137 is a microphone provided in the mobile terminal 130, and by inputting voice from the voice input unit 137, the control unit 132 performs voice recognition based on the voice signal from the voice input unit 137 and creates a signal corresponding to the content of the voice.

[0095] The audio output unit 138 is a speaker provided in the mobile terminal 130, and generates audio based on an audio signal from the control unit 132. The audio input unit 137 and the audio output unit 138 can be replaced by earphones with a microphone, headphones, or the like connected to the mobile terminal 130.

[0096] The audio output unit 138 preferably uses a pair of left and right stereo speakers, stereo earphones, or stereo headphones, and outputs audio in a so-called 3D surround format.

[0097] Vibrator 139 is a vibration generating means provided in mobile terminal 130, and generates vibrations based on an activation signal from control unit 32. The function of vibrator 139 to generate vibrations is called a vibration function.

[0098] Furthermore, the mobile terminal 130 can connect to the operation server 30B via the wireless network 110, download the app 150 for realizing the method of supporting pedestrians and others using voice guidance of the present invention, and install it in the memory unit 133, thereby receiving the data necessary for supporting pedestrians and others using voice guidance from the operation server 30B via the wireless network 110.

[0099] Furthermore, if the operation server 30B is equipped with a self-location estimation unit 39, the user takes a photo with the mobile terminal 130, transfers the photo to the operation server 30B, and the operation server 30B searches for the photo and matches it with images in the database, extracts the user's self-location information, and transfers the self-location information 126 from the operation server 30B to the mobile terminal 130.

[0100] Specifically, the mobile terminal 130 captures, for example, a two-dimensional still image using the imaging unit 135, and the control unit 132a of the mobile terminal 130 transmits the captured image signal 135b to the operation server 30B via the transmission / reception unit 131 and the network 110. The network 110 is connected to the operation server 30B via a wireless network of any configuration for mobile phones, such as 4G or 5G, that can be connected to the transmission / reception unit 131. The transmission / reception unit 131 may be connected via a dedicated line network (VPN) or the Internet. The operation server 30B may be a so-called cloud.

[0101] Unlike the operation server 30 in Figure 1, the operation server 30B shown in Figure 8 further includes a self-location estimation unit 39 connected to the control unit 33B, and an application 120 for the self-location estimation unit 39 stored in the memory unit 32B.

[0102] The map data 31 stored in the navigation server 30B is data including a point cloud that links three-dimensional image data acquired by a three-dimensional lidar with position information acquired by a position sensor such as a GPS and an attitude detection sensor such as an IMU. Here, the point cloud is a point cloud displayed in absolute coordinates by adding position information, that is, coordinates based on absolute coordinates (latitude, longitude, altitude), to a three-dimensional point cloud that indicates the relative distance to an object ahead. A point cloud may also be simply called a point cloud. The absolute coordinates may further include the time of position acquisition by a GPS sensor, and the absolute coordinates may be expressed as (latitude, longitude, altitude, time).

[0103] According to the evacuation guidance system 100A of Fig. 8, the operation server 30B includes a self-location estimation unit 39 that estimates the self-location of the user of the mobile terminal 130. The self-location estimation unit 39 generates a data set 225 (see Fig. 9) consisting of a two-dimensional point cloud and location information using map data 31 stored in the storage unit 32B. Upon receiving a two-dimensional search image for self-location estimation from the mobile terminal 130, the self-location estimation unit 39 estimates self-location information 26 of the search image using the map data 31 and the data set 25, and transmits the self-location information 26 to the mobile terminal 30B via a network 40B including a wireless network. Specifically, the self-location estimation unit 39 of the operation server 30B extracts features of the search image and the map data image, matches the features of the search image with the features of the image in the map data 31, extracts features of the image in the map data 31 that match the features of the search image, and estimates the user's location from the image in the map data 31 extracted by matching and the data set, thereby extracting self-location information 126 of the user.

[0104] 9, in the operation server 30B, the map data 31 is used for self-location estimation as a file of a data set 225. For example, the file of the data set 225 of the robot operating system is created in the file format of a ROSBAG 225a for storing data of messages used in the robot operating system.

[0105] Since the map data 31 includes three-dimensional image data acquired by a lidar and position information acquired by a position sensor and an IMU, the dataset 225 is configured to run a so-called LIOSAM (Lidar Inertial Odometry via Smoothing and Mapping) algorithm using a lidar.

[0106] The dataset 225 is organized into an organized folder structure 225e containing a folder 225b (Interactive-SLAM pose-graph folder) related to LIOSAM, image data 225c, and a CSV file (imu-data.csv) 225d related to IMU data acquired by ROS topic echo from a posture detection sensor such as an IMU. The organized folder structure 225e generates a color point cloud 225g and position information data 225h using a position dataset generator 25f.

[0107] The workflow diagram shown in Figure 10 is an example of self-location estimation, and shows an algorithm executed in the operation server 30B by the application 120 for the self-location estimation unit 39 stored in the memory unit 32B and the data set 225 generated from the map data 32.

[0108] The self-location estimation unit 39 extracts features from the image of the three-dimensional map data 31 stored in the storage unit 32B and the image signal 135b of the two-dimensional image taken by the user's camera, which serves as the search image, and extracts an image of the map data 31 that matches the search image from the features, thereby performing six-dimensional self-location estimation of the search image. This algorithm is also called hierachical localization (Hloc).

[0109] 10, when a two-dimensional query image for self-location estimation is received from the mobile terminal 130 (query image in step C1), features of the query image are extracted in step C2, recorded in step C3, and recorded as a global descriptor in step C4. Here, the features of the search image in step C2 are extracted by so-called machine learning such as a convolution neural network (also called CNN), for example.

[0110] Meanwhile, the actual three-dimensional map data 31 stored in the storage unit 32B is converted into two-dimensional dataset images in step D1, features of the dataset images are extracted in step D2, and the extracted features of the dataset images are recorded as a global descriptor index in step D3. Here, the features of the dataset images are extracted by so-called machine learning such as a convolution neural network (also called a CNN).

[0111] Next, in step E1, the global descriptors (see C4) of the search image are searched for using the index of the global descriptors of the dataset images to find the closest image, and in step E2, the image with the highest score is extracted. In step E3, the position of the user's camera is estimated from the extracted dataset image using PnP (a method for determining the camera position from a three-dimensional dataset image and a two-dimensional search image). Next, in step E4, the user's position information (six-dimensional data relating to position and orientation (6DoF pose result)) is estimated and sent as the user's self-position information 126 to the mobile terminal 130 via the network 110. Hereinafter, the estimation of the user's self-position information 126 will be described in more detail using an example. [Example]

[0112] In the present embodiment, the support for pedestrians and the like by the audio guide is carried out by estimating the user's own position information 126 using the mobile terminal 30 and the server 30B described below. Mobile terminal 30: iPhone (manufactured by Apple, model number: iPhone 13 Pro) (iPhone is a registered trademark, hereinafter referred to as iPhone.) Operation server 30B: AWS, Amazon's cloud service, was used as the server. An application 150 for a method of supporting pedestrians and the like using audio guidance was installed on the mobile terminal 130, and map data 31, the application 150, and software related to Figures 9 and 10 were installed on the AWS to configure an operation server 30B including a self-position estimation unit 39 and the like. Actual three-dimensional map data 31 stored in memory unit 32B was created by the inventor and includes a map of the inside of a building 40. It was confirmed that when a user takes a photo using an iPhone 13 Pro, which is the mobile terminal 130, the operation server 30B transmits the user's self-position information 126 to the iPhone 13 Pro at extremely high speed in about 5 seconds.

[0113] FIG. 11 shows a two-dimensional photograph taken by a user as a search image, and a two-dimensional image extracted by feature matching from the three-dimensional point cloud of the operation server 30B. As shown by the white circles in Figure 11, the matching points in the search image on the left closely match the matching points in the two-dimensional image extracted from the three-dimensional point cloud in the center. In the present invention, the accuracy of self-location information 126 was high, both indoors and outdoors, and was within 20 cm. It was also confirmed that various landmarks such as walls and objects could be recognized. Thus, with the evacuation guidance system 100A of the present invention, image recognition and point cloud orientation calculation are performed using images captured by the camera on the user's mobile device 130 and three-dimensional images stored on the navigation server 30B, allowing self-location accuracy of within 20 cm even in environments where a position sensor such as GPS is not used.

[0114] (Comparative Example) As a comparison, Table 1 shows the results of a comparison with the voice guidance provided by Google Maps. In this case, a search image taken by the user is matched with an image from Street View. The accuracy of the self-location information was around 60 cm outdoors, and the error increased even more indoors. While the voice guide provided by Google Maps does not allow users to know at all which floor of the building they are on, the present invention allows users to use the elevators installed in the building and also displays information on the floors where they need to get on and off.

[0115] [Table 1] Note that the self-location estimation method can also be achieved by recognizing tags attached to a 3D point cloud map. In rare cases, this method may be used partially in featureless environments, such as long, all-white corridors. The accuracy of a 3D point cloud map is 2 to 5 mm, and autonomous vehicles using 3D point cloud maps have high steering accuracy, so the self-location accuracy is within 10 cm. Regarding self-location accuracy, there is a large error in the user's walking control, and it varies depending on the user, but accuracy improves if the user holds the mobile terminal 130 correctly while walking.

[0116] According to the present invention, a three-dimensional point cloud is used as map data 31, and self-location estimation is performed by matching feature points of a search image acquired by a camera image with feature points of the point cloud of map data 31. This makes it possible to recognize walls and objects in front of the user with a high accuracy of 20 cm or less, both indoors and outdoors.

[0117] (Evacuation guidance using the voice guide of the present invention) The control unit 33 of the operation server 30B notifies the user of the location of the fire using a fire notification signal through a voice guide and outputs a voice from the speaker of the mobile terminal to guide the user to an emergency elevator. In fire mode, the control unit 33 of the operation server 30B cannot control the emergency elevator control unit, so it guides the user to the location of the emergency elevator via the management server 30. When the user gets on the emergency elevator, the user can press the destination floor button to move the emergency elevator to the ground floor, i.e., the first floor. When the emergency elevator arrives at the first floor, the emergency elevator stops, and the user gets off on the first floor and moves to a safe location. Furthermore, in the event of an earthquake in a building, the elevator control unit 44 and the operation server 30B work together to notify the user of the evacuation route through a voice guide, guide the user to an evacuation route using the elevator 41, and move the user to a safety confirmation location.

[0118] (Displayed using AR) By superimposing an image of a person taken by the self-position estimation unit 39 of the present invention and the person's position estimated from this image on the display of a mobile terminal, it is possible to provide accurate evacuation guidance to able-bodied people. This is so-called AR navigation (Augmented Reality Navigation), and by superimposing virtual information on real space, it is possible to provide not only route guidance but also information needed for evacuation guidance in emergencies such as fires and earthquakes in a more intuitive manner.

[0119] Figure 12 is a photograph showing an example of the AR navigation display. Figure 12 is a photograph showing an example of the display screen of an iPhone 13 Pro. When a user is staying in a parking lot in the basement of a building, the user is shown a route guide saying, "Please use stairs C to go up to B1F," and the arrow on the screen indicates that stairs C is on the left. In the event of a fire or earthquake, this type of AR navigation can be used to show evacuation routes and provide evacuation guidance.

[0120] The present invention can be implemented in various forms without departing from the spirit of the invention. For example, in the above-described embodiment, the mobile terminal 30 is described as a smart device such as a smartphone or a tablet, but it is clear that the present invention is not limited to this and may be a mobile terminal such as a notebook computer.

[0121] According to the evacuation guidance system using an autonomous vehicle of the present invention, the person's own position can be accurately estimated using image data captured using the mobile device of the person inside the building 40.Therefore, in the event of a fire occurring inside the building 40, the fire alarm equipment and the operation server 30 of the autonomous vehicle 20 will work together to guide the person along the evacuation route using audio guidance, and move them to a safety confirmation position.

[0122] In the above-described embodiment, the control unit 32 of the mobile terminal 30 creates the travel route 32d and the evacuation route based on the map data 31, but this is not limited to this. The server 30 may create the evacuation route 132d for a person based on the destination information 132a and current location information 132b of the mobile terminal 130.

[0123] If the autonomous vehicle 20 is a passenger vehicle that transports users at public transportation stations, airports, etc., it may be equipped with a display device with a touch panel function that allows the user to use the autonomous vehicle 20, a camera that reads a QR code (registered trademark) to authenticate the user, a scanner, a short-range wireless NFC (Near Field Communication) transceiver, etc.

[0124] The evacuation guidance system using the autonomous vehicle of the present invention may be used for daily inspection work. Images acquired by the imaging unit 66b while the autonomous vehicle 20 is traveling during normal operations and / or images acquired by a surveillance camera installed in the building 40 may be transmitted to the operation server 30. The operation server 30 performs AI analysis of the acquired images. If an obstacle that impedes passage through the evacuation route or facility equipment 48 is detected, the operation server 30 may issue an alert to the management server 30 of the building 40. When an alert is issued, a disaster prevention officer can rush to the site to inspect the facility equipment 48 and remove the obstacle. This allows a disaster prevention officer to detect obstacles within the building 40 from images acquired by the autonomous vehicle 20 and quickly eliminate obstacles during evacuation guidance, without the need for on-site inspection. [Explanation of symbols]

[0125] 20 Self-driving vehicles 21 Main body 22 wheels 23 Display section 23a,23b opening 24 Surveillance Cameras 25 Speaker (first audio output unit) 26 Storage section 27 Light 28 Turn signal 29 Control Unit 30,30B operation server 31 Map Data 32 Storage section 32a Connection database 32b analytical database 32c Driving management database 32d Database for remote monitoring and / or remote operation 33 Control Unit 34 Analysis Department 36 Driving Management Department 37 Collaboration Department 38 Remote monitoring and / or remote control unit 39 Self-position estimation part 40 Buildings 41 Elevator 41a Speaker (third audio output unit) 42 Migration Path 43 Elevator car 43a Speaker (second audio output unit) 43b Display (second image output unit) 43c Camera (Elevator Analysis Sensor) 43e crew information 44 Elevator control unit 45 Management Server 45a, 45b Gateway 46 Elevator Hall 46a Elevator Door 46b Elevator movement display 46c Push Button 46e Display (third image output unit) 48 Facility equipment control unit 48a Facility equipment usage information 48b Facility equipment control information 50 Network 52 Fire alarm system 53 Receiver 54a First transmission line 54b Second transmission line 55 Fire detector 56 Transmitter 57 District sound equipment 58 Fire Door 59 Fire shutter 61 CPU 61a Current location information 61b Destination information 61c Travel route 61d Elevator Information 61e Elevator control information 61F Elevator Usage Statistics 61g Home probability information 61h Driving route data 62 Battery 63 Display 64 Memory section 65 Drive unit 65a motor 65b driver 66 Detector 66a Attitude detection sensor 66b Monitoring sensor (imaging unit) 66c Distance Sensor 66d Position Sensor 66e Bumper Sensor 66f Status transmission / reception unit 70 display screen 75 Model Data Screen 76th floor of the building 100,100A Evacuation Guidance System Using Autonomous Driving Vehicles (Evacuation Guidance System) 110 Wireless Network 120 Application for self-location estimation unit 39 126 Self-location information 130 Mobile Devices 131 Transmitter / Receiver 132,132a Control unit 133 Storage section 134 Display section 135 Imaging unit 135b Imaging signal 136 Detector 136a Attitude detection sensor 136b Position Sensor 136d Ambient Sensor 136e Height sensor 137 Audio input section 138 Audio output section 139 Vibrator 225 datasets 225a ROSBAG 225b LIOSAM folder 225c Image data 225d IMU data CSV file (imu-data.csv) 225e Organized Folder Structure 225f Location Dataset Generator 225g color point cloud 225h location data S1~S3 (S1~S4) Detection signal

Claims

1. an autonomous vehicle operation server having a memory unit in which three-dimensional map data relating to roads, surrounding environments, facility equipment including elevators, and buildings equipped with fire alarm equipment is registered; the fire alarm system connected to the operation server via a network; the autonomously driven vehicle connected to the operation server via the network; a facility equipment control unit connected to the operation server via the network and controlling the facility equipment based on facility equipment usage information and facility equipment control information from the autonomously driven vehicle; Including, The autonomous driving vehicle includes a drive unit, a control unit, an imaging unit, a detection unit including an attitude detection sensor and a position sensor, a transmission / reception unit connected to the network, and an audio output unit. Equipped with the operation server stores in the storage unit an evacuation route that has been created in advance based on the map data; The fire alarm system outputs a fire alarm when a fire signal from a fire detector installed in the building is received by a receiver, and outputs a fire information signal including the location of the fire to the operation server; When the operation server receives the fire information signal from the fire alarm system, it displays the location of the fire on map data of the facility, compares it with a pre-created evacuation route, creates an evacuation route including a safe elevator, and transmits the created evacuation route to the autonomously driven vehicle; An evacuation guidance system using an autonomous vehicle, in which the control unit of the autonomous vehicle outputs evacuation guidance audio to people in the vicinity from the audio output unit.

2. 2. The evacuation guidance system using an autonomous vehicle according to claim 1, wherein the facility equipment includes an opening / closing device such as a security door, a gate, an automatic door, or a shutter, and the elevator.

3. 2. The evacuation guidance system using an autonomous vehicle according to claim 1, wherein the autonomous vehicle detects operation information of facility equipment including the elevator while moving from its current location to a safety confirmation location, and sends the operation information to the operation server.

4. 2. The evacuation guidance system using an autonomous vehicle according to claim 1, wherein a control unit of the autonomous vehicle controls boarding and disembarking of the elevator and opening and closing of the opening and closing device while the autonomous vehicle is moving from the current position to the safety confirmation position.

5. 2. The evacuation guidance system using an autonomous vehicle as described in claim 1, wherein the control unit of the autonomous vehicle outputs an announcement from the audio output unit of the autonomous vehicle to notify people in the vicinity of the autonomous vehicle of the evacuation guidance of the autonomous vehicle while the autonomous vehicle is traveling along the evacuation route, and outputs an announcement from the audio output unit of the autonomous vehicle regarding the use of facility equipment, including elevators, based on usage information of the facility equipment, including elevators.

6. 2. The evacuation guidance system using an autonomous vehicle as described in claim 1, wherein the operation server remotely monitors and / or remotely operates the operation of the autonomous vehicles based on the current position of the autonomous vehicles and the evacuation route, and the relative positions of the autonomous vehicles and each other or the autonomous vehicles and the surrounding environment.

7. The building is provided with a management server that controls the entire building, 7. The evacuation guidance system using an autonomous vehicle according to claim 6, wherein the management server controls a control unit of the facility equipment including the elevator based on usage information of the facility equipment including the elevator.

8. 2. The evacuation guidance system using an autonomous vehicle according to claim 1, wherein the imaging unit includes a camera or LIDAR capable of length measurement, a computer that processes image signals acquired by the camera or LIDAR, and a storage device, and acquires AI analysis information based on image signals including fire, smoke, and people acquired along the evacuation route, and outputs the AI ​​analysis information to a control unit of facility equipment including the elevator and to the operation server.

9. 2. The evacuation guidance system using an autonomous vehicle according to claim 1, wherein the imaging unit of the autonomous vehicle includes an infrared camera, a computer that processes image signals acquired by the infrared camera, and a storage device, and acquires AI analysis information based on image signals including fire, smoke, and people acquired along the evacuation route, and outputs the AI ​​analysis information to a control unit of facility equipment including the elevator and to the operation server.

10. 10. The evacuation guidance system using an autonomous vehicle according to claim 8 or 9, wherein the operation server receives the AI ​​analysis information, modifies the evacuation route as necessary, and sends the modified evacuation route information to one or more autonomous vehicles in the building and / or surrounding environment.

11. 10. The evacuation guidance system using an autonomous vehicle according to claim 8 or 9, wherein the operation server uses AI analysis to make decisions including counting the number of passengers in the elevator, adjusting evacuation routes, and resetting evacuation routes from images acquired by an imaging unit of the autonomous vehicle or a surveillance camera installed in the building.

12. 2. The evacuation guidance system using an autonomous vehicle as described in claim 1, wherein, when the operation server receives elevator stop information due to an earthquake from the elevator control unit and / or a management server that controls the building, it transmits the elevator stop information to the autonomous vehicle, creates the evacuation route so that the autonomous vehicle and surrounding people get off the elevator at the nearest floor to where the elevator will stop, and provides evacuation guidance to notify people around the autonomous vehicle of the occurrence of an earthquake and the suspension of elevator operation.

13. 2. The evacuation guidance system using an autonomous vehicle as described in claim 1, wherein the autonomous vehicle further comprises a display unit, and evacuation guidance based on information about evacuation guidance from the operation server and / or the building management server is displayed on the display unit.

14. 2. An evacuation guidance system using an autonomous vehicle as described in claim 1, wherein the operation server is equipped with a self-location estimation unit that identifies the current location of a person from image data transmitted from a mobile device carried by the person, and when the operation server receives the fire information signal, transmits the estimated current location and evacuation route of the person to the mobile device, and the control unit of the mobile device outputs the current location and evacuation route as audio and / or displays it on a display unit.

15. 2. The evacuation guidance system using an autonomous vehicle as described in claim 1, wherein images acquired by the imaging unit while the autonomous vehicle is traveling during normal operations or images acquired by a surveillance camera installed in the building are transmitted to the operation server, the operation server performs AI analysis of the acquired images, and if an obstacle that impedes passage along the evacuation route or the facility equipment is detected, the operation server notifies an alert to a management server of the building.

16. 2. An evacuation guidance system using an autonomous vehicle as described in claim 1, wherein the autonomous vehicle is a work vehicle for various purposes such as delivery, security, cleaning, disinfection, reception guidance, and advertising, a passenger vehicle for transporting users such as the elderly, injured, or disabled, or an autonomous delivery vehicle for delivering various items such as food, drink, and merchandise.

Images