Autonomous driving system, autonomous driving method, and autonomous driving program

Abstract

To provide an autonomous traveling system, an autonomous traveling method, and an autonomous traveling program capable of improving the work efficiency of a worker who generates a traveling route for an autonomous traveling device.SOLUTION: An autonomous traveling device 10 comprises: a recognition processing unit 413 that recognizes a worker based on an image captured by a camera 43; a travel processing unit 418 that causes the autonomous traveling device 10 to travel while following the worker; a detection processing unit 415 that detects a predetermined gesture operation of the worker based on the captured image; a registration processing unit 414 that registers action information in which a specific action corresponding to a specific gesture operation and position information of the autonomous traveling device 10 are associated with each other when the detected gesture operation is a preset specific gesture operation; and a generation processing unit 416 that generates a traveling route based on locus information of a traveling locus along which the autonomous traveling device 10 travels following the worker and action information registered by the registration processing unit 414.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an autonomous driving system, an autonomous driving method, and an autonomous driving program for driving an autonomous driving device capable of autonomous driving. [Background technology]

[0002] In recent years, labor shortages and rising labor costs have led to a shortage of workers to guard (surveillance) and clean facilities such as shopping malls, train stations, and airports. As a result, there has been an increasing introduction of mobile robots (autonomous driving devices), such as industrial autonomous security robots and cleaning robots, which are designed to be autonomous and safe.

[0003] Some of these types of autonomous mobile devices are equipped with a teaching function that stores a travel route based on the operator's operation (see, for example, Patent Document 1). For example, the operator sets the autonomous mobile device to a route teaching mode and drives the autonomous mobile device in a work area to travel a desired route, storing the trajectory traveled by the autonomous mobile device as a travel route. When set to the autonomous mobile device in the autonomous mobile mode, the autonomous mobile device reproduces and travels the travel route stored in the route teaching mode. This makes it possible for the autonomous mobile device to travel autonomously according to a travel route generated based on the operator's operation (teaching operation). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-182175 Summary of the Invention [Problem to be solved by the invention]

[0005] However, with conventional technology, an operator must manually move the autonomous driving device while performing operations such as setting the driving route on an operating terminal.If the operator is unfamiliar with the operation, the operation of setting the driving route (teaching operation) takes time, resulting in a problem of reduced work efficiency.

[0006] An object of the present invention is to provide an autonomous driving system, an autonomous driving method, and an autonomous driving program that can improve the work efficiency of an operator who generates a driving route for an autonomous driving device. [Means for solving the problem]

[0007] According to one aspect of the present invention, there is provided an autonomous driving system for driving an autonomous driving device based on a driving route. The autonomous driving system includes: a recognition processing unit that recognizes an operator based on detection information from a detection unit mounted on the autonomous driving device; a driving processing unit that drives the autonomous driving device to drive while following the operator; a detection processing unit that detects a predetermined registration operation of the operator based on the detection information; a registration processing unit that, when the registration operation detected by the detection processing unit is a predetermined specific registration operation, registers action information that associates a specific action corresponding to the specific registration operation with position information of the autonomous driving device; and a generation processing unit that generates the driving route based on trajectory information of a driving route taken by the autonomous driving device while following the operator and the action information registered by the registration processing unit.

[0008] According to another aspect of the present invention, there is provided an autonomous driving method for driving an autonomous driving device based on a driving route. In the autonomous driving method, one or more processors execute the following steps: a recognition step of recognizing an operator based on detection information from a detection unit mounted on the autonomous driving device; a driving step of driving the autonomous driving device while following the operator; a detection step of detecting a predetermined registration operation of the operator based on the detection information; a registration step of registering, if the registration operation detected in the detection step is a predetermined specific registration operation, motion information associating a specific motion corresponding to the specific registration operation with position information of the autonomous driving device; and a generation step of generating the driving route based on trajectory information of a driving route taken by the autonomous driving device while following the operator and the motion information registered in the registration step.

[0009] An autonomous driving program according to another aspect of the present invention is a program for causing an autonomous driving device to drive based on a driving route. The autonomous driving program is a program for causing one or more processors to execute the following steps: a recognition step of recognizing an operator based on detection information from a detection unit mounted on the autonomous driving device; a driving step of causing the autonomous driving device to drive while following the operator; a detection step of detecting a predetermined registration operation of the operator based on the detection information; a registration step of registering, if the registration operation detected in the detection step is a predetermined specific registration operation, motion information that associates a specific action corresponding to the specific registration operation with position information of the autonomous driving device; and a generation step of generating the driving route based on trajectory information of a driving route taken by the autonomous driving device while following the operator and the motion information registered in the registration step. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide an autonomous driving system, an autonomous driving method, and an autonomous driving program that can improve the work efficiency of an operator who generates a driving route for an autonomous driving device. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a perspective view showing the external appearance of the front side of an autonomous mobile device according to an embodiment of the present invention. [Figure 2] FIG. 2 is a perspective view showing the appearance of the rear side of the autonomous mobile device according to the embodiment of the present invention. [Figure 3] FIG. 3 is a front view of the autonomous driving device according to the embodiment of the present invention. [Figure 4] FIG. 4 is a functional block diagram showing the configuration of an autonomous driving device according to an embodiment of the present invention. [Figure 5] FIG. 5 is a diagram showing an example of map information registered in the storage unit of the autonomous mobile device according to the embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of teaching information registered in the storage unit of the autonomous driving device according to the embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of gesture information registered in the storage unit of the autonomous mobile device according to the embodiment of the present invention. [Figure 8] FIG. 8 is a diagram showing how the autonomous mobile device according to the embodiment of the present invention travels while following a worker. [Figure 9] FIG. 9 is a diagram showing a teaching operation according to an embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 12] FIG. 12 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 13] FIG. 13 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 14] FIG. 14 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 15]FIG. 15 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 16] FIG. 16 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 17] FIG. 17 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 18] FIG. 18 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. [Figure 19] FIG. 19 is a diagram illustrating an example of operation information registered in the storage unit of the autonomous mobile device according to the embodiment of the present invention. [Figure 20] FIG. 20 is a flowchart showing an example of a driving route generation process executed by an autonomous driving device according to an embodiment of the present invention. [Figure 21] FIG. 21 is a flowchart showing an example of a driving route generation process executed by an autonomous driving device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Note that the following embodiment is an example of the present invention, and does not limit the technical scope of the present invention.

[0013] [Autonomous Driving Device 10] FIG. 1 is a perspective view showing the appearance of the front side of an autonomous mobile device 10 according to an embodiment of the present invention, and FIG. 2 is a schematic view showing the appearance of the rear side of the autonomous mobile device 10. In the following description, the up-down direction D1, the front-back direction D2, and the left-right direction D3 shown in each figure will be used. The autonomous mobile device 10 is an example of an autonomous mobile device of the present invention.

[0014] The autonomous mobile device 10 is an autonomous mobile device that moves by autonomously traveling within a facility such as a shopping mall, a station, or an airport, and is also referred to as a mobile robot. The autonomous mobile device 10 is used for various purposes, such as a security robot, a cleaning robot, a guide robot, a care robot, and a luggage transport robot. For example, the autonomous mobile device 10 guards (monitors) a security target area within a facility while traveling by autonomous traveling. In addition, for example, the autonomous mobile device 10 cleans a cleaning target area while traveling by autonomous traveling. In addition, for example, the autonomous mobile device 10 guides a user to a destination or displays guidance information or advertising information on the display panel 21 while traveling by autonomous traveling. The autonomous mobile device 10 autonomously travels based on various setting information, such as a preset traveling route, a time period, and a return location to return to for charging.

[0015] In the following, a security robot equipped with a security function will be described as an example of the autonomous mobile device 10 of this embodiment. For example, in a shopping mall, the autonomous mobile device 10 performs a travel process along a preset travel route during a predetermined time period (e.g., at night), while also performing a security process to detect the presence or absence of an abnormality such as a suspicious person.

[0016] The autonomous driving device 10 includes a device main body 11 and various functional units provided in the device main body 11. Specifically, the device main body 11 includes a driving unit 12, a motor (not shown), a battery (not shown), an operation unit 20 (see FIG. 2), a display panel 21 (see FIG. 1), a charging connection unit 30 (see FIG. 2), and a control unit 40 (control device).

[0017] As shown in Fig. 1, device main body 11 has an exterior cover 11A that forms the exterior of the device. Furthermore, as shown in Fig. 2, device main body 11 has a chassis 11B at its bottom. Chassis 11B is provided generally parallel to the floor. Furthermore, a support frame is appropriately provided inside device main body 11 to support the above-mentioned functional units.

[0018] The running unit 12 is provided at the bottom of the device body 11. The running unit 12 transmits a conveying force in the forward direction to the floor surface while maintaining the running posture of the device body 11, and is attached to the chassis 11B. The running unit 12 has a pair of wheels 121 and four casters 122 for running.

[0019] The wheel 121 is located in the center of the chassis 11B in the front-to-rear direction, and is rotatably supported at both ends in the left-right direction D3 (width direction). The four casters 122 are used to maintain the traveling posture of the device main body 11, and are rotatably supported at both ends of the front end of the chassis 11B and both ends of the rear end of the chassis 11B. When the autonomous mobile device 10 is placed on a floor surface, the outer circumferential surfaces of the wheels 121 and casters 122 are supported by the floor surface. This allows the device main body 11 to be maintained in the traveling posture shown in FIGS. 1 and 2.

[0020] An output shaft of the motor is connected to the rotation shaft of the wheel 121 via a transmission mechanism such as a reduction gear. Therefore, when the motor is driven and its rotational driving force is output from the output shaft, the rotational driving force of the motor is transmitted to the wheel 121. In this embodiment, the motor is provided individually for each of a pair of wheels 121. Therefore, the rotational speed of each wheel 121 is controlled by individually driving and controlling each motor. For example, when the rotational speed of each wheel 121 is controlled to be uniform, the autonomous mobile device 10 moves straight or backward, and when the rotational speed of each wheel 121 is controlled to be different speeds, the autonomous mobile device 10 turns toward the wheel 121 with the slower rotational speed.

[0021] The battery is provided in the center of the device body 11. The battery supplies driving power to the motor.

[0022] As shown in Fig. 2, a rear cover 17 that covers the rear surface is provided on the rear surface of the device main body 11. A charging connection unit 30 that is used when charging the battery is provided on the rear cover 17. The charging connection unit 30 has three power receiving terminals 31 that are connected to three power supply terminals provided on a charging station. Note that if the autonomous mobile device 10 is a cleaning robot, components related to the cleaning function are arranged inside the rear cover 17.

[0023] 3 is a front view of the autonomous mobile device 10. As shown in FIG. 3, a front laser sensor 41, a sonar sensor 42, and a camera 43 are provided on the front side of the autonomous mobile device 10.

[0024] The front laser sensor 41 is provided in a groove 175 extending in the width direction and formed in the lower part of the front surface of the device main body 11. The front laser sensor 41 is disposed in the center of the groove 175. The front laser sensor 41 includes a laser emitting element, a laser driver for driving the laser emitting element, a light receiving element, and a light receiving processing circuit for converting the output of the light receiving element into a digital signal. The front laser sensor 41 is connected to and controlled by the control unit 40. The front laser sensor 41 scans a laser beam forward in the width direction (horizontal direction) within a predetermined scanning angle (e.g., 120°). When the front laser sensor 41 receives the laser beam reflected by an object (target object) that it illuminates, the control unit 40 measures the time it takes for the laser beam to return and calculates the distance to the object at each scanning position based on the measured value. This allows the control unit 40 to determine the distance and position of an object located in front of the autonomous mobile device 10 (in the direction of travel), as well as the shape and size of the object in the width direction.

[0025] The sonar sensor 42 is provided below the display panel 21. A sonar sensor 42 is provided at each end in the width direction on the front of the device main body 11. The sonar sensor 42 is connected to and controlled by the control unit 40. The sonar sensor 42 detects an object using sound waves, and measures the distance to the object based on the time it takes for the sound waves to reflect off the object and return.

[0026] The camera 43 is provided above the display panel 21. The camera 43 is arranged at the center of the width direction on the front side of the device body 11. The camera 43 is connected to the control unit 40 and is controlled by the control unit 40. The camera 43 captures an image of a predetermined range in front of it (for example, a range of 120 degrees horizontally and 100 degrees vertically) at a predetermined frame rate and outputs the captured image data to the control unit 40. The control unit 40 sequentially acquires the image data from the camera 43 and performs predetermined image processing (described later). The camera 43 is an example of a detection unit of the present invention, and also an example of a camera of the present invention.

[0027] Additionally, side laser sensors 45 are provided on both sides of the device body 11. The side laser sensors 45 are configured similarly to the front laser sensor 41, including a laser emitting element, a laser driver for driving the laser emitting element, a light receiving element, and a light receiving processing circuit for converting the output of the light receiving element into a digital signal. The side laser sensors 45 are connected to and controlled by the control unit 40. The side laser sensors 45 scan with laser light from the front, downward, and rearward within a predetermined scanning angle (e.g., 180°). When the side laser sensors 45 receive the laser light reflected by an object (target), the control unit 40 measures the time it takes for the laser light to return and calculates the distance to the object at each scanning position based on the measured value. This allows the control unit 40 to grasp the distance and position of objects present in front of the autonomous mobile device 10 (in the direction of travel), as well as steps and obstacles on the floor, and their shapes and sizes in the scanning direction.

[0028] A surveillance camera 44 is also provided on the top surface of the device body 11. The surveillance camera 44 is located in the center of the top panel of the device body 11. The surveillance camera 44 is connected to and controlled by the control unit 40. The surveillance camera 44 is configured to be able to change its imaging range and captures a predetermined imaging range in accordance with commands from the control unit 40. For example, the surveillance camera 44 is equipped with a rotation mechanism that can rotate the lens 360 degrees horizontally and 180 degrees vertically, and rotates the lens in a direction instructed by the control unit 40 to capture an image of a predetermined range in that direction. The surveillance camera 44 captures an image of a predetermined range at a predetermined frame rate and outputs the captured image data to the control unit 40. The control unit 40 sequentially acquires the image data from the surveillance camera 44 and performs predetermined image processing (described below).

[0029] The autonomous mobile device 10 autonomously travels along the predetermined travel route by using information on a map (environmental map) corresponding to the security area and detection information from the front laser sensor 41, sonar sensor 42, camera 43, surveillance camera 44, and side laser sensor 45. Note that the camera 43 and the surveillance camera 44 may be configured as a single camera. For example, the surveillance camera 44 provided on the top surface of the device body 11 may serve as the camera 43.

[0030] The operation unit 20 (see FIG. 2) is provided at the top of the back surface of the device body 11. The operation unit 20 is attached to the exterior cover 11A. The operation unit 20 is a device operated by an operator, and is, for example, a device having a touch panel that allows touch operation. The operation unit 20 displays an operation screen that allows the operator to perform various operations (teaching operation, registration operation, setting operation, travel instruction operation, etc.). Operation information for the operation unit 20 is transferred to the control unit 40 and used for travel control by the control unit 40. The operation unit 20 may also be provided on the top surface of the device body 11.

[0031] The display panel 21 (see FIG. 1) is provided on the front of the device main body 11. The display panel 21 is, for example, a liquid crystal panel. Various operation screens (see FIGS. 10 to 18) are displayed on the display panel 21 by the control unit 40 during teaching operations. In addition, announcement information such as security information is displayed on the display panel 21 by the control unit 40 while the autonomous mobile device 10 is autonomously traveling (on guard duty). The security information is, for example, information that announces a warning or a report when a suspicious person is detected. The display panel 21 is an example of a display unit of the present invention.

[0032] The operating handle 22 is provided at the top of the back surface of the device main body 11. The operating handle 22 is attached to the exterior cover 11A. The operating handle 22 is an operating member that is held by an operator when the operator manually drives and operates the autonomous mobile device 10.

[0033] The communication unit 25 (see FIG. 4) is a communication interface that connects the autonomous mobile device 10 to a network via a wired or wireless connection and performs data communication with an external device such as a server (not shown) via the network in accordance with a predetermined communication protocol.

[0034] The storage unit 50 (see FIG. 4) is a non-volatile storage unit such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that stores various types of information. Specifically, data such as map information 51, teaching information 52, gesture information 53, and operation information 54 is stored in the storage unit 50. FIG. 5 is a diagram showing an example of the map information 51. FIG. 6 is a diagram showing an example of the teaching information 52. FIG. 7 is a diagram showing an example of the gesture information 53. The operation information 54 (see FIG. 19) will be described later.

[0035] As shown in FIG. 5, the map information 51 registers information about an environmental map corresponding to a security area (work area) in which the autonomous mobile device 10 travels. One or more environmental maps are registered in the map information 51. FIG. 5 illustrates one environmental map M1. For example, if an environmental map is generated for each floor of a facility, multiple environmental maps corresponding to the respective floors of the facility are registered in the map information 51. Specifically, an environmental map M1 for floor F1 of the facility, an environmental map M2 for floor F2, and an environmental map M3 for floor F3 are registered in the map information 51. The environmental maps may be generated in advance by an external device, or may be generated based on the distance to and position of an obstacle detected by each sensor (such as the front laser sensor 41 and the sonar sensor 42) while the autonomous mobile device 10 is traveling. In other words, the control unit 40 may generate an environmental map of the travel area by traveling the autonomous mobile device 10. The following description will be given using the environmental map M1 shown in FIG. 5 as an example.

[0036] As shown in FIG. 6, the teaching information 52 registers information about the teaching operation by the operator.

[0037] In the teaching operation according to this embodiment, worker A, who is in charge of teaching the driving route of the facility corresponding to the environmental map M1 shown in FIG. 5, walks within the facility, and the autonomous mobile device 10 drives, following behind worker A, as shown in FIG. 8. FIG. 9 shows worker A performing the teaching operation and the autonomous mobile device 10 driving, following worker A. Reference symbol P0 in FIG. 9 indicates the start position P1 of the teaching operation. While driving, following worker A, the autonomous mobile device 10 acquires various information and registers it in the teaching information 52.

[0038] Specifically, information such as "position information," "gesture detection," and "gesture type" is registered in the teaching information 52 for each predetermined time interval. "Position information" is information indicating the position (coordinates) of the autonomous mobile device 10. The position information is calculated based on the environmental map M1 and the detection results of the front laser sensor 41, the sonar sensor 42, the camera 43, etc. "Gesture detection" is information indicating whether the autonomous mobile device 10 has detected a gesture operation by operator A. The gesture operation is a predetermined movement of operator A's hands (right and left hands), face, or whole body. For example, the gesture operation may be a movement of raising the right hand, a movement of raising the left hand, a pointing movement, a movement of waving the hand sideways or vertically, or a movement of bending the waist or knees to lower one's posture (bending). When the control unit 40 analyzes the image captured by the camera 43 and detects the gesture operation, it registers "1" in "gesture detection" in the teaching information 52. "Gesture type" is information indicating the movement of the operator corresponding to the gesture operation. During the teaching operation, worker A performs a predetermined gesture operation at a predetermined position while the autonomous mobile device 10 is following the autonomous mobile device 10. When the control unit 40 detects the gesture operation by worker A, it registers the motion information of the gesture operation in the "gesture type."

[0039] As shown in FIG. 7, information regarding the gesture operation by the operator is registered in the gesture information 53. Specifically, information such as "gesture type" and "operation content" is registered in the gesture information 53. Various pieces of information in the gesture information 53 are set in advance by an administrator or the like and stored in the storage unit 50. The "gesture type" is the type of gesture operation that is to be recognized as the gesture operation. The control unit 40 is able to recognize the gesture operation registered in the gesture type. The "operation content" is information indicating the operation content that the autonomous mobile device 10 is to execute when the control unit 40 recognizes the gesture operation.

[0040] The gesture type "pointing" is associated with and registered as an action content of "registering an action of capturing an image of the pointing direction with a monitoring camera for the duration of the gesture operation." For example, during a teaching operation, when control unit 40 detects a "pointing" gesture operation by worker A based on an image captured by camera 43, control unit 40 registers information about the action of causing monitoring camera 44 to capture an image of the pointing direction for the duration of the pointing operation, in association with the position information of autonomous mobile device 10 (action information 54 in FIG. 19 ).

[0041] Furthermore, the gesture type "wave" is registered in association with the action content of "registering the action of capturing 360-degree images with a surveillance camera for the duration of the gesture operation." For example, when control unit 40 detects a gesture operation of "wave" by worker A based on an image captured by camera 43 during a teaching operation, control unit 40 registers information about the action of causing surveillance camera 44 to capture images in a 360-degree direction for the duration of the hand-waving operation in association with the position information of autonomous mobile device 10 (action information 54 in FIG. 19 ).

[0042] Furthermore, the gesture type "bend" is registered in association with the action content of "registering an action to display specific information only for the duration of the gesture operation." For example, during a teaching operation, when control unit 40 detects a "bend" gesture operation by worker A based on an image captured by camera 43, control unit 40 registers information about the action to display specific information on display panel 21 in association with the position information of autonomous mobile device 10 (action information 54 in FIG. 19 ).

[0043] Furthermore, the gesture type "raise right hand" is registered in association with the action content of "transition to action information registration mode." For example, during a teaching operation, when the control unit 40 detects a gesture operation of "raise right hand" by the operator A based on the image captured by the camera 43, the control unit 40 transitions to a registration mode in which action content associated with a specific gesture operation (such as "pointing," "waving," or "bending") is registered in association with the position information of the autonomous mobile device 10. When the control unit 40 detects the specific gesture operation after transitioning to the registration mode, the control unit 40 registers information on the action content associated with the gesture operation in association with the position information of the autonomous mobile device 10 (action information 54 in FIG. 19 ).

[0044] Furthermore, the gesture type "raise left hand" is registered in association with the operation content of "end teaching operation." For example, when the control unit 40 detects the gesture operation of "raise left hand" by the worker A based on the image captured by the camera 43 during the teaching operation, the control unit 40 ends the teaching operation and ends the registration process of the operation content and the registration process of the position information.

[0045] In this way, when the control unit 40 detects a gesture operation by the worker A, it refers to the gesture information 53 and executes the operation content associated with the detected gesture operation.

[0046] In another embodiment, some or all of the information such as the map information 51, teaching information 52, gesture information 53, and operation information 54 may be stored in a data server accessible from the autonomous mobile device 10 via a network.

[0047] Furthermore, the storage unit 50 stores control programs such as a driving route generation program for causing the control unit 40 to execute a driving route generation process (see FIGS. 20 and 21) described below. For example, the driving route generation program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a reading device (not shown) such as a CD drive or DVD drive provided in the autonomous mobile device 10 and stored in the storage unit 50.

[0048] The control unit 40 is provided on the top of the device main body 11 (not shown). FIG. 4 is a functional block diagram showing the configuration of the control unit 40. The control unit 40 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various types of arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS that cause the CPU to execute various types of arithmetic processing are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various types of information and is used as a temporary storage memory (work area) for the various types of processing executed by the CPU. The control unit 40 controls the autonomous driving device 10 by having the CPU execute various control programs that are stored in advance in the ROM or the storage unit 50.

[0049] Specifically, as shown in Fig. 4, the control unit 40 includes various processing units such as an acquisition processing unit 411, a display processing unit 412, a recognition processing unit 413, a registration processing unit 414, a detection processing unit 415, a generation processing unit 416, a notification processing unit 417, and a driving processing unit 418. The control unit 40 functions as the various processing units by executing various processes in accordance with the driving route generation program using the CPU. Some or all of the processing units may be configured with electronic circuits. The driving route generation program may be a program for causing multiple processors to function as the processing units.

[0050] The acquisition processing unit 411 acquires image data of captured images captured by the camera 43 during the teaching operation. Specifically, as shown in Fig. 8, during the teaching operation, while the autonomous mobile device 10 is traveling following the operator A, the camera 43 captures images of the area ahead at a predetermined frame rate and sequentially outputs the image data of the captured images to the control unit 40. The acquisition processing unit 411 repeatedly acquires image data from the camera 43 during the teaching operation.

[0051] The display processing unit 412 displays various types of information on the display panel 21. Specifically, the display processing unit 412 displays operation screens (see FIGS. 10 to 18) related to teaching operations on the display panel 21. The display processing unit 412 supports the teaching operation by the worker A by displaying announcement information on each operation screen.

[0052] The recognition processing unit 413 recognizes the operator based on the captured image corresponding to the image data acquired by the acquisition processing unit 411. Specifically, when starting a teaching operation, operator A captures an image of himself / herself with the camera 43 of the autonomous mobile device 10. As shown in FIG. 10 , the display processing unit 412 displays the captured image captured by the camera 43 on the display panel 21. Note that the display processing unit 412 may also display information (environmental map information) about the area (environmental map M1) targeted for the teaching operation, a travel route ID, a travel route name, and the like on the operation screen.

[0053] When the recognition processing unit 413 detects a person from the captured image, it displays the detected person's image to confirm whether it is indeed worker A (see FIG. 11). When worker A nods in front of the camera 43, the recognition processing unit 413 recognizes the detected person as the worker who will perform the teaching operation. When the recognition processing unit 413 recognizes the worker, the control unit 40 starts the teaching operation (see FIG. 12). When the teaching operation starts, the display processing unit 412 displays the starting position P0 of the teaching operation, the current position of the autonomous mobile device 10 (marked with a "★"), and the current position of worker A (marked with a "●") on the environmental map M1, as shown in FIG. 12.

[0054] When the teaching operation is started, the control unit 40 controls the traveling of the autonomous mobile device 10 so that it follows the recognized worker A. Specifically, the control unit 40 controls the traveling speed and traveling direction of the autonomous mobile device 10 based on the position of the worker A contained in the captured image acquired from the camera 43. The display processing unit 412 displays the traveling trajectory of the autonomous mobile device 10 on the operation screen while the autonomous mobile device 10 is traveling and following the worker A (see FIG. 13).

[0055] If the recognition processing unit 413 is no longer able to recognize (loses sight of) operator A after the teaching operation has started, the notification processing unit 417 displays a message (warning information) indicating that recognition is not possible on the operation screen (see FIG. 14). The notification processing unit 417 may output the message as sound, or may output a predetermined warning sound. In other words, the notification processing unit 417 issues warning information when the autonomous mobile device 10 is no longer able to follow operator A.

[0056] In this way, the recognition processing unit 413 recognizes the worker based on the captured image acquired from the camera 43. In another embodiment, the recognition processing unit 413 may recognize the worker based on the detection results of a sensor (e.g., the front laser sensor 41, the sonar sensor 42, etc.) mounted on the autonomous mobile device 10. The sensor is an example of a detection unit of the present invention.

[0057] The registration processing unit 414 registers the position information of the current position of the autonomous mobile device 10 during the teaching operation in the teaching information 52 (see FIG. 6).

[0058] The detection processing unit 415 detects a predetermined gesture operation (registered operation of the present invention) of worker A based on the captured image. Specifically, the detection processing unit 415 identifies the gesture operation of worker A included in the captured image by referring to a plurality of gesture operations (gesture types) registered in gesture information 53 (see FIG. 7). When the detection processing unit 415 detects the predetermined gesture operation, the control unit 40 executes a specific operation (operation content in FIG. 7) associated with the gesture operation.

[0059] Here, for example, when operator A stops at an arbitrary position for a predetermined time during the teaching operation, the display processing unit 412 displays the operation screen shown in FIG. 15. Specifically, the display processing unit 412 displays announcement information regarding a gesture operation (e.g., "raise right hand") that transitions to a registration mode in which a specific action is registered, and a gesture operation (e.g., "raise left hand") that terminates the teaching operation. When the detection processing unit 415 detects the specific gesture operation of "raise right hand" by operator A from the image captured by the camera 43, the control unit 40 transitions to a registration mode in which the specific action content is associated with the position information of the autonomous mobile device 10 and registered. Note that when the detection processing unit 415 detects "raise right hand," the registration processing unit 414 registers information related to "raise right hand" in the teaching information 52 (time t12 in FIG. 6).

[0060] Furthermore, upon transitioning to the registration mode, if the gesture operation detected by detection processing unit 415 is a specific gesture operation, registration processing unit 414 registers operation information that associates the operation content corresponding to the specific gesture operation with position information of the autonomous mobile device 10. Specifically, if the gesture operation detected by detection processing unit 415 is a first gesture operation, registration processing unit 414 registers first operation information that causes the autonomous mobile device 10 to perform a first operation at a first position, and if the gesture operation detected by detection processing unit 415 is a second gesture operation, registration processing unit 414 registers second operation information that causes the autonomous mobile device 10 to perform a second operation at a second position.

[0061] In addition, the registration processing unit 414 registers the operation information when the gesture operation detected by the detection processing unit 415 is a specific gesture operation and when a registration operation (e.g., a nodding operation) is received from the worker in response to the information of the specific gesture operation displayed on the display panel 21.

[0062] For example, when worker A performs a pointing operation at an arbitrary position, the display processing unit 412 displays inquiry information for confirming the gesture operation on the operation screen shown in FIG. 16. When worker A nods toward the camera 43, the control unit 40 executes a specific action associated with the pointing operation. Here, the registration processing unit 414 executes processing to associate "an action of capturing an image in the pointing direction with a monitoring camera for the duration of the gesture operation (pointing operation)" with the current position [x10, y10] of the autonomous mobile device 10 and register it (time t13 in FIG. 6). In addition, the display processing unit 412 displays the registration content on the operation screen shown in FIG. 17. This allows worker A to recognize that the intended specific action has been reliably registered.

[0063] For example, when worker A performs a hand-waving operation at an arbitrary position, the display processing unit 412 displays inquiry information on the operation screen to confirm the gesture operation. When worker A nods toward the camera 43, the control unit 40 executes a specific action associated with the hand-waving operation. Here, the registration processing unit 414 executes processing to associate and register "the action of capturing 360 degrees of images with a surveillance camera for the duration of the gesture operation (hand-waving)" with the current position [x20, y20] of the autonomous mobile device 10 (time t23 in FIG. 6).

[0064] The generation processing unit 416 generates the travel path based on trajectory information of the travel trajectory of the autonomous mobile device 10 traveling while following the worker A and the operation information registered by the registration processing unit 414. Specifically, the generation processing unit 416 generates the trajectory information shown in FIG. 18 based on position information registered in the teaching information 52 (see FIG. 6). The trajectory information includes information such as a position P0 at which autonomous traveling starts, a route R1 along which autonomous traveling follows, and a position G at which autonomous traveling ends. The display processing unit 412 displays the generated trajectory information on the operation screen shown in FIG. 18 to prompt the worker A to confirm it. The display processing unit 412 may also display positions P1 and P2 of the specific operation on the operation screen shown in FIG. 18. Note that position P1 corresponds to position information [x10, y10] in FIG. 6, and position P2 corresponds to position information [x20, y20] in FIG. 6.

[0065] The registration processing unit 414 also registers operation information 54 based on the teaching information 52 shown in FIG. 6. FIG. 19 shows an example of the operation information 54. Information indicating operation information when the autonomous mobile device 10 is caused to autonomously travel is registered in the operation information 54. For example, operation information when the autonomous mobile device 10 starts autonomous travel is associated with and registered in position information [x1, y1]. Operation information of a specific operation registered by operator A using a specific gesture operation (pointing) is associated with and registered in position information [x10, y10]. Operation information of a specific operation registered by operator A using a specific gesture operation (waving) is associated with and registered in position information [x20, y20]. Operation information of a specific operation when the autonomous mobile device 10 ends autonomous travel is associated with and registered in position information [x50, y50]. The generation processing unit 416 generates a travel path including the travel trajectory and the operation information 54. The generation processing unit 416 stores the generated travel route information in the storage unit 50 in association with the travel route ID and the travel route name.

[0066] For example, the memory unit 50 stores information on a driving route (driving route ID "0001") corresponding to an environmental map M1 on floor F1 of the facility, information on a driving route (driving route ID "0002") corresponding to an environmental map M2 on floor F2 of the facility, and information on a driving route (driving route ID "0003") corresponding to an environmental map M3 on floor F3 of the facility.

[0067] In addition, the generation processing unit 416 may generate a map (environmental map) based on images captured by the camera 43 while the autonomous mobile device 10 is traveling following the worker A, and generate the traveling route on the map.

[0068] Here, the travel trajectory of the autonomous mobile device 10 traveling while following worker A corresponds to the movement trajectory of worker A, and therefore may be diagonal or zigzag with respect to the passageway. Therefore, the generation processing unit 416 may generate the travel path by correcting the travel trajectory to a path that corresponds to the traveling operation of the autonomous mobile device 10. For example, the generation processing unit 416 may correct the travel path to a straight line, or correct the travel path so that a right-left turn route makes a 90-degree turn. Furthermore, the generation processing unit 416 may generate a travel path by correcting the movement trajectory of worker A to a position that can avoid obstacles, etc., in the security area.

[0069] The driving processing unit 418 performs autonomous driving according to the driving route generated by the generation processing unit 416. For example, a facility manager operates the operation unit 20 (see FIG. 2) to select a driving route (e.g., a driving route ID and a driving route name) along which the autonomous driving will occur. When a driving route is selected, the driving processing unit 418 reads information about the driving route (see FIGS. 18 and 19) from the storage unit 50, and performs operations according to the operation information 54 shown in FIG. 19 while autonomously driving according to the route R1 shown in FIG. 18. As a result, the autonomous mobile device 10 starts autonomous driving, for example, at position P0, and when it reaches position P1[x10, y10], it stops temporarily and causes the monitoring camera 44 to take images in a 90-degree direction to the left for 30 seconds. Thereafter, the autonomous mobile device 10 travels according to route R1, and when it reaches position P2[x20, y20], it stops temporarily and causes the monitoring camera 44 to take images for 90 seconds while rotating 360 degrees. Thereafter, the autonomous mobile device 10 travels along the route R1, and ends its autonomous traveling when it reaches the position G[x50, y50].

[0070] In this way, when the autonomous mobile device 10 of this embodiment is a security robot, the autonomous mobile device 10 autonomously travels along the travel route in accordance with the registered contents of the teaching operation by worker A, and guards the area to be guarded by performing the specific operations at positions P1 and P2.

[0071] [Route generation process] 20 and 21, the driving route generation process executed in the autonomous mobile device 10 will be described. Specifically, in this embodiment, the driving route generation process is executed by the control unit 40 of the autonomous mobile device 10.

[0072] The present invention can be understood as an invention of a driving route generation method that executes one or more steps included in the driving route generation process. Furthermore, one or more steps included in the driving route generation process described herein may be omitted as appropriate. The steps in the driving route generation process may be executed in a different order as long as the same operational effect is achieved. Furthermore, while the description here takes as an example a case where the control unit 40 executes each step in the driving route generation process, another embodiment can also be considered as a driving route generation method in which one or more processors execute each step in the driving route generation process in a distributed manner.

[0073] When the control unit 40 receives an instruction from the operator to start the teaching operation, it executes the travel route generation process.

[0074] First, in step S11, the control unit 40 acquires a captured image from the camera 43. For example, when the worker A performs an operation to start a teaching operation on the autonomous mobile device 10, the control unit 40 outputs a shooting instruction to the camera 43 and acquires the captured image captured by the camera 43.

[0075] When the control unit 40 acquires the captured image from the camera 43 (S11: Yes), it detects a person from the captured image in step S12. When the control unit 40 detects a person (S12: Yes), it displays an image of the detected person on the display panel 21 in step S13 (see FIG. 10).

[0076] Next, in step S14, the control unit 40 displays an inquiry screen on the display panel 21 inquiring whether the person displayed is the worker (see FIG. 11). When the worker nods in front of the camera 43, the control unit 40 obtains the worker's confirmation and recognizes the detected person as the worker who will perform the teaching operation (S14: Yes). The control unit 40 repeats the processing of steps S1 to S4 until it recognizes the worker who will perform the teaching operation.

[0077] When the control unit 40 recognizes (determines) the worker who will perform the teaching operation (S14: Yes), it executes the teaching process in step S15 (see FIG. 12). When the control unit 40 starts the teaching process, it detects the worker from the captured image acquired by the camera 43 in step S16 (see FIG. 13). If the control unit 40 cannot detect the worker from the captured image acquired by the camera 43 (S16: No), it displays warning information indicating that the worker cannot be recognized on the operation screen in step S161 (see FIG. 14).

[0078] When the control unit 40 detects a worker from the captured image acquired by the camera 43 (S16: Yes), it acquires position information of the autonomous mobile device 10 while the autonomous mobile device 10 does not stop for a predetermined time (S17: No) (S171). The control unit 40 registers the acquired position information in the teaching information 52 (see FIG. 6). In this way, the control unit 40 accumulates position information of the movement trajectory of the autonomous mobile device 10 while the autonomous mobile device 10 recognizes and follows the worker while traveling (see FIG. 6).

[0079] Here, when the operator stops at any position and the autonomous mobile device 10 stops for a predetermined time (S17: Yes), in step S18, the control unit 40 displays announcement information on the operation screen shown in FIG. 15 regarding a gesture operation (e.g., "raise right hand") that will transition to a registration mode for registering a specific action, and a gesture operation (e.g., "raise left hand") that will end the teaching operation.

[0080] Next, in step S19, the control unit 40 determines whether or not a gesture operation of "raising the right hand" by the worker has been detected from the captured image acquired from the camera 43. If the control unit 40 detects a gesture operation of "raising the right hand" (S19: Yes), the process proceeds to step S20. On the other hand, if the control unit 40 does not detect a gesture operation of "raising the right hand" (S19: No), the process proceeds to step S191.

[0081] In step S20, the control unit 40 transitions to a registration mode in which a specific action to be performed by the autonomous mobile device 10 is registered.

[0082] Next, in step S21, the control unit 40 determines whether the gesture operation detected from the captured image acquired by the camera 43 is a specific gesture operation. If the gesture operation is a specific gesture operation (S21: Yes), the process proceeds to step S22. If the gesture operation is not a specific gesture operation (S21: No), the process returns to step S19. Note that the specific gesture operation here is a gesture operation for registering a specific action to be performed when the autonomous mobile device 10 autonomously travels, such as "pointing," "waving," or "bending."

[0083] In step S22, the control unit 40 displays inquiry information for confirming the detected specific gesture operation on the operation screen shown in FIG.

[0084] Next, in step S23, the control unit 40 determines whether or not a registration operation of motion information corresponding to the specific gesture operation has been accepted from the worker. For example, when the worker nods towards the camera 43 (see FIG. 16), the control unit 40 accepts the registration operation of motion information corresponding to the specific gesture operation.

[0085] Next, in step S24, the control unit 40 registers operation information corresponding to the specific gesture operation. For example, if the specific gesture operation at the current position [x10, y10] of the autonomous mobile device 10 is "pointing," the control unit 40 registers "an operation of capturing an image in the pointing direction with a monitoring camera for the duration of the pointing operation" in association with the position [x10, y10] (time t13 in FIG. 6). Also, for example, if the specific gesture operation at the current position [x20, y20] of the autonomous mobile device 10 is "waving," the control unit 40 registers "an operation of capturing an image in 360 degrees with a monitoring camera for the duration of the waving" in association with the position [x20, y20] (time t23 in FIG. 6). The control unit 40 registers operation information related to the specific operation in the operation information 54 (see FIG. 19). Furthermore, the control unit 40 displays the registered content on the operation screen shown in FIG. 17. After the control unit 40 registers the operation information, the process returns to step S15. Returning to step S15, the control unit 40 causes the autonomous mobile device 10 to travel following the operator.

[0086] Returning to step S15, when the autonomous mobile device 10 travels following the worker and detects the specific gesture operation "waving" at position [x20, y20], the control unit 40 registers "an action of capturing 360 degrees of footage with a surveillance camera for the duration of waving" in association with position [x20, y20] (time t23 in FIG. 6). The control unit 40 registers the action information regarding the specific action in the action information 54 (see FIG. 19).

[0087] Here, in step S191, if the control unit 40 detects a gesture operation of "raising the left hand" by the worker (S191: Yes), the control unit 40 ends the teaching process (S192). Thereafter, the control unit 40 generates a driving route based on trajectory information (see FIG. 18) of the driving trajectory along which the autonomous mobile device 10 travels while following the worker and the registered operation information 54 (see FIG. 19). The control unit 40 stores the generated driving route information in the storage unit 50.

[0088] In this way, the control unit 40 generates a driving route for the autonomous mobile device 10. When an operator causes the autonomous mobile device 10 to drive autonomously, the operator selects a desired driving route from one or more driving routes (route patterns) registered in the storage unit 50. The control unit 40 causes the autonomous mobile device 10 to drive autonomously according to the driving route selected by the operator.

[0089] As described above, the autonomous mobile device 10 according to this embodiment recognizes a worker based on detection information (e.g., captured images) from a detection unit (e.g., camera 43) mounted on the autonomous mobile device 10. The autonomous mobile device 10 then drives while following the recognized worker. The autonomous mobile device 10 then detects a predetermined registration operation (e.g., a gesture operation) by the worker based on the detection information, and, if the detected registration operation is a predetermined specific registration operation, registers action information that associates the specific action corresponding to the specific registration operation with position information of the autonomous mobile device 10. The autonomous mobile device 10 then generates a driving route for autonomous driving based on trajectory information of the driving trajectory along which the autonomous mobile device drives while following the worker and the registered action information.

[0090] According to this configuration, in the teaching operation, the operator can register a travel route by moving (walking) along the route on which the operator wants the autonomous mobile device 10 to travel autonomously. Furthermore, when the operator wants the autonomous mobile device 10 to perform a specific action at a predetermined position on the travel route, the operator can register the specific action by performing a predetermined gesture operation in the teaching operation. In this way, the operator does not need to set or register a travel route using an operation terminal or the like, and can easily perform the teaching operation. Therefore, even an operator who is unfamiliar with teaching operations can perform the teaching operation. Furthermore, the work efficiency of the teaching operation can be improved.

[0091] The method for registering the specific action in the teaching operation is not limited to a gesture operation using the worker's body. For example, the specific action may be registered using the worker's voice. For example, if the worker utters the words "Take a surveillance camera take a picture of the area 90 degrees to the left for 30 seconds" at the position [x10, y10], the control unit 40 acquires the voice and registers the content of the voice as a specific action in association with the position [x10, y10].

[0092] In the above-described embodiment, the autonomous mobile device 10 is a security robot, but the autonomous mobile device of the present invention may also be, for example, a cleaning robot. When the autonomous mobile device is a cleaning robot, the autonomous mobile device performs cleaning work while autonomously traveling along the generated travel path. The autonomous mobile device also performs specific operations at positions P1 and P2. Here, the specific operations may include, for example, an operation to focus on cleaning work at position P1 and an operation to spray chemicals to remove dirt at position P2.

[0093] In another embodiment, the autonomous mobile device of the present invention may be, for example, a guide robot. When the autonomous mobile device is a guide robot, the autonomous mobile device autonomously travels along the generated travel route and performs a specific action at positions P1 and P2. Here, the specific action may be, for example, an action to display a specific advertisement according to the position, or an action to announce advertising information by voice, etc.

[0094] In the above-described embodiment, the autonomous mobile device 10 alone corresponds to the autonomous mobile system according to the present invention. However, the autonomous mobile system according to the present invention may include one or more of the autonomous mobile device 10 and a server (information processing device). For example, if multiple components of the autonomous mobile device 10 and the server cooperate to share and execute the driving path generation process, a system including the multiple components that execute the process can be considered as the autonomous mobile system according to the present invention. For example, the server alone may constitute the autonomous mobile system according to the present invention. Specifically, the server may include each processing unit (acquisition processing unit 411, display processing unit 412, recognition processing unit 413, registration processing unit 414, detection processing unit 415, generation processing unit 416, notification processing unit 417, and driving processing unit 418) of the control unit 40 shown in FIG. 4 and control the autonomous mobile device 10. The server may also store map information 51, teaching information 52, gesture information 53, and operation information 54. The server may be configured as a physical server or a virtual server (cloud server). [Explanation of symbols]

[0095] 10: Autonomous driving device 21: Display panel 40: Control section 51: Map information 52: Teaching Information 53: Gesture information 54: Operation information 411: Acquisition processing unit 412: Display processing unit 413: Recognition processing section 414: Registration processing unit 415: Detection processing unit 416: Generation processing unit 417: Notification processing unit 418: Driving processing unit

Claims

1. An autonomous driving system that drives an autonomous driving device based on a driving route, a recognition processing unit that recognizes an operator based on detection information from a detection unit mounted on the autonomous mobile device; a travel processing unit that causes the autonomous driving device to travel while following the worker; a detection processing unit that detects a predetermined registration operation by the worker based on the detection information; a registration processing unit that, when the registration operation detected by the detection processing unit is a predetermined first specific registration operation, transitions to a registration mode in which a specific action is registered, and, when the registration operation detected by the detection processing unit after transitioning to the registration mode is a predetermined second specific registration operation, registers action information in which the specific action corresponding to the second specific registration operation is associated with position information of the autonomous mobile device; a generation processing unit that generates the travel path based on trajectory information of a travel trajectory along which the autonomous mobile device travels while following the worker and the operation information registered by the registration processing unit; a display processing unit that displays the detection information on a display unit mounted on the autonomous driving device; Equipped with the driving processing unit stops the autonomous driving device when the worker stops at an arbitrary position, The display processing unit displays information about the first specific registration operation for transitioning to the registration mode on the display unit when the autonomous driving device stops for a predetermined period of time.

2. the registration processing unit does not execute the registration process of the operation information when the mode is not shifted to the registration mode. The autonomous driving system according to claim 1 .

3. When a third specific registration operation is detected by the detection processing unit, the generation processing unit generates the travel route based on the trajectory information and the action information acquired until the third specific registration operation is detected. The autonomous driving system according to claim 1 or 2.

4. the detection processing unit detects a predetermined gesture operation by the worker, When the gesture operation detected by the detection processing unit is a predetermined specific gesture operation, the registration processing unit registers the operation information in which the specific operation corresponding to the gesture operation is associated with position information of the autonomous driving device. The autonomous driving system according to any one of claims 1 to 3.

5. the registration processing unit registers first operation information that causes the autonomous mobile device to perform a first operation at a first position when the gesture operation detected by the detection processing unit is a first gesture operation, and registers second operation information that causes the autonomous mobile device to perform a second operation at a second position when the gesture operation detected by the detection processing unit is a second gesture operation; The autonomous driving system according to claim 4 .

6. the registration processing unit registers the operation information when the registration operation detected by the detection processing unit is the second specific registration operation and a registration operation is received from the worker in response to the detection information displayed on the display unit.

6. The autonomous driving system according to claim 1.

7. the generation processing unit generates a map based on the detection information detected by the detection unit while the autonomous mobile device is traveling following the worker, and generates the traveling route on the map. The autonomous driving system according to any one of claims 1 to 6.

8. the generation processing unit corrects the traveling trajectory to a path corresponding to the traveling operation of the autonomous traveling device to generate the traveling path. The autonomous driving system according to any one of claims 1 to 7.

9. further comprising a notification processing unit that issues warning information when the autonomous mobile device becomes unable to follow the worker; The autonomous driving system according to any one of claims 1 to 8.

10. An autonomous driving method for driving an autonomous driving device based on a driving route, comprising: one or more processors, a recognition step of recognizing an operator based on detection information from a detection unit mounted on the autonomous mobile device; a traveling step of causing the autonomous mobile device to travel while following the worker; a detection step of detecting a predetermined registration operation by the worker based on the detection information; a registration step of transitioning to a registration mode for registering a specific action when the registration operation detected in the detection step is a predetermined first specific registration operation, and registering action information that associates a specific action corresponding to the second specific registration operation with position information of the autonomous mobile device when the registration operation detected in the detection step after transitioning to the registration mode is a predetermined second specific registration operation; a generating step of generating the travel path based on trajectory information of a travel trajectory along which the autonomous mobile device travels while following the worker and the operation information registered in the registering step; a display step of displaying the detection information on a display unit mounted on the autonomous driving device; Run In the traveling step, stopping the autonomous traveling device when the operator stops at an arbitrary position; In the display step, when the autonomous driving device has stopped for a predetermined period of time, information regarding the first specific registration operation for transitioning to the registration mode is displayed on the display unit.

11. An autonomous driving program that causes an autonomous driving device to travel based on a travel route, a recognition step of recognizing an operator based on detection information from a detection unit mounted on the autonomous mobile device; a traveling step of causing the autonomous mobile device to travel while following the worker; a detection step of detecting a predetermined registration operation by the worker based on the detection information; a registration step of transitioning to a registration mode for registering a specific action when the registration operation detected in the detection step is a predetermined first specific registration operation, and registering action information that associates a specific action corresponding to the second specific registration operation with position information of the autonomous mobile device when the registration operation detected in the detection step after transitioning to the registration mode is a predetermined second specific registration operation; a generating step of generating the travel path based on trajectory information of a travel trajectory along which the autonomous mobile device travels while following the worker and the operation information registered in the registering step; a display step of displaying the detection information on a display unit mounted on the autonomous driving device; on one or more processors, In the traveling step, stopping the autonomous traveling device when the operator stops at an arbitrary position; an autonomous driving program that, in the display step, displays information on the display unit regarding the first specific registration operation for transitioning to the registration mode when the autonomous driving device has stopped for a predetermined period of time.

Images