Autonomous driving device and method for evaluating its validity

Abstract

To improve the accuracy of reproduction travel without installing an initialization object in a reproduction travel path in advance or accurately disposing a device body at a start position.SOLUTION: An autonomous travel device 1 includes: a device body 2; a travel unit 3; a measurement unit 6; a map creation unit 21; a teaching travel control unit 22 and a reproduction travel control unit 23 that control the travel unit 3 such that when a teaching travel mode is executed, the device body 2 creates and stores a travel path from a teaching start position to a teaching end position and a work travel map, and when a reproduction travel mode is executed, the device body 2 at a reproduction start position is caused to travel; an imaging unit 7 that captures an environment image; and a validity evaluation unit 25 that acquires, as a teaching-period environment image, the environment image captured during the execution of the teaching travel mode, acquires, as a current environment image, the environment image captured during the execution of the reproduction travel mode, and evaluates validity of the reproduction start position by performing a first matching process in which the teaching-period environment image and the current environment image are compared with each other.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to an autonomous traveling device capable of traveling in response to manual operation and automatic operation, and a validity evaluation method for evaluating the validity of a reproduction start position in reproduction traveling of the autonomous traveling device.

Background Art

[0002] Conventionally, an autonomous traveling device creates or sets and stores an environmental map, a traveling route, traveling settings and work settings (cleaning settings) in accordance with a program input in advance, and performs autonomous traveling and automatic work (cleaning), that is, automatic traveling work (automatic traveling cleaning), based on the stored results, and thus has a function capable of executing reproduction traveling. For example, the autonomous traveling device is applied to industrial (commercial) autonomous traveling devices that clean the floor surfaces of work areas such as commercial facilities, offices, hotels, and hospitals.

[0003] For example, Patent Document 1 discloses a technology for initializing a robot (autonomous traveling device) to autonomously travel along a route, in which the robot detects an initialization object, then determines the position of the robot relative to the initialization object, then learns the route through a user's demonstration, associates an action along the route with the position relative to the initialization object, thereafter detects the initialization object again later, determines the position of the robot relative to the initialization object, then autonomously navigates along the learned route, and executes the action associated with the position relative to the initialization object.

Prior Art Literature

Patent Literature

[0004]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0005] In conventional technologies such as those described in Patent Document 1, it is necessary to pre-install initialization objects along the path in order to initialize the robot (autonomous mobile device) so that it can determine its position in the environment relatively quickly. However, if the autonomous mobile device repeatedly performs replicate runs along the same path, it is necessary to keep the initialization objects permanently installed along the path, which can damage the aesthetics of the environment, including the path. On the other hand, if the initialization objects are installed along the path each time a replicate run is performed and then collected after the replicate run is completed, the installation and collection of the initialization objects becomes time-consuming and the work becomes complicated. Furthermore, it is difficult to accurately install the initialization objects in the same position each time a replicate run is performed, and if there is a large deviation in the installation position of the initialization objects, it may hinder the execution of the replicate run.

[0006] The present invention has been made in view of the above-mentioned problems, and the object of the present invention is to provide an autonomous driving device and a method for evaluating the validity of the same that can improve the accuracy of a simulated drive without having to pre-install initialization objects along the simulated drive route or accurately position the device body at the starting position. [Means for solving the problem]

[0007] To solve the above problems, the first autonomous driving device of the present invention is an autonomous driving device capable of driving based on manual operation input and autonomous driving based on automatic operation input, comprising: a device body; a driving unit that drives the device body; a measuring unit that measures the distance and angle between the device body and objects around the device body; a map creation unit that creates a local map of the area around the device body by coordinate transformation of the distance and angle measured by the measuring unit; and, when the teaching driving mode is executed, while the driving unit drives the device body in response to manual operation input, the measuring unit and the map creation unit are operated to connect the local maps, thereby determining the driving path and the driving path of the device body from the teaching start position to the teaching end position. The system comprises: a driving control unit that creates and stores a work driving map of the surrounding area of ​​the driving route, and controls the driving unit to drive the device body at the reproduction start position according to the driving route and the work driving map when the reproduction driving mode is executed; an imaging unit provided on the device body that captures an environmental image showing the surrounding area of ​​the device body; an evaluation unit that acquires the environmental image captured by the imaging unit when the teaching driving mode is executed as the teaching environmental image, acquires the environmental image captured by the imaging unit when the reproduction driving mode is executed as the current environmental image, and performs a first matching process comparing the teaching environmental image and the current environmental image to evaluate the validity of the reproduction start position. The evaluation unit acquires the local map created by the map creation unit at the start of the teaching driving mode as the teaching local map, and acquires the local map created by the map creation unit at the start of the reproduction driving mode as the current local map. It then performs a second matching process comparing the teaching local map and the current local map, and evaluates the validity of the reproduction start position based on the first matching process and the second matching process. It is characterized by the following:

[0008] According to the first autonomous driving device of the present invention, even if the operator mistakenly selects a work route map for a cleaning area different from the cleaning area in which the autonomous driving device is located, when the autonomous driving device performs a reproduction run, it is possible to avoid situations where the device stops due to an error because it is unable to continue the work due to following the wrong work route map, thereby maintaining good accuracy in the reproduction run. Furthermore, when performing a reproduction run, if the operator does not position the autonomous driving device at the correct teaching start position, it is possible to avoid situations where the device stops due to an error because it is unable to continue the work due to following the wrong work route map, thereby maintaining good accuracy in the reproduction run. Therefore, it is possible to improve the accuracy of the reproduction run without having to pre-install initialization objects along the reproduction run route or precisely position the device body at the teaching start position. Furthermore, according to the first autonomous driving device of the present invention, technically reliable image matching processing technology can be applied not only to environmental images but also to local maps to evaluate the validity of the reproduction start position and the work driving map. In addition, since the same image matching processing technology can be applied to both environmental images and local maps, the overall processing can be simplified and the processing speed can be improved.

[0009] To solve the above problems, the second autonomous driving device of the present invention is characterized in that the evaluation unit outputs a first evaluation score as numerical information based on the agreement rate between the teaching environment image and the current environment image obtained by the first matching process.

[0010] According to the second autonomous driving device of the present invention, it is possible to accurately verify the selection of the work driving map using technically reliable image matching processing technology. Furthermore, the operator can easily verify the validity of the reproduction start position and the work driving map using numerical information as a result of the matching processing.

[0013] To solve the above problems, the fourth autonomous driving device of the present invention is characterized in that the evaluation unit outputs a total evaluation score as numerical information, which is obtained by combining a first evaluation score based on the agreement rate between the teaching environment image and the current environment image obtained by the first matching process, and a second evaluation score based on the agreement rate between the teaching local map and the current local map obtained by the second matching process.

[0014] According to the fourth autonomous driving device of the present invention, it becomes possible to comprehensively verify the validity of the reproduction start position and the work driving map from multiple perspectives, making the validity evaluation results more accurate and making it easier to understand the evaluation results, thereby reducing the burden on the operator.

[0015] To solve the above problems, the fifth autonomous driving device of the present invention acquires a plurality of teaching environment images by capturing the environment images with the imaging unit at predetermined travel distance intervals during a predetermined start section after starting to drive in the teaching driving mode, and acquires each of the local maps created by the map creation unit as a plurality of teaching local maps. The evaluation unit, when executing the reproduction driving mode, if the overall match rate based on the first match rate by the first matching process and the second match rate by the second matching process at the reproduction start position is less than a predetermined match rate threshold, acquires a plurality of current environment images by capturing the environment images with the imaging unit at predetermined travel distance intervals during a predetermined start section after starting to drive in the reproduction driving mode, and acquires each of the local maps created by the map creation unit as a plurality of current local maps. The evaluation unit evaluates that the validity of the reproduction start position is satisfied if the overall match rate based on the first match rate by the first matching process of the plurality of current environment images and the second match rate by the second matching process of the plurality of current local maps at the predetermined start section is equal to or greater than a predetermined match rate threshold.

[0016] According to the fifth autonomous driving device of the present invention, even when working while repeatedly zigzagging within the same cleaning area, the amount of data to be stored can be reduced compared to storing environmental images and local maps over the entire travel path, because the matching of environmental images and local maps is concentrated in the initial start period. Furthermore, even if the selection of the cleaning area is correct, but the reproduction start position and direction (angle) are considerably off, the reproduction drive can be started as is, and the validity of the reproduction start position and work travel map can be evaluated. Moreover, if the validity of the work travel map is evaluated after the reproduction drive has started, the reproduction drive can be continued as is, so the reproduction drive can be easily started even without precisely positioning the reproduction start position of the autonomous driving device at the teaching start position.

[0017] To solve the above problems, in the sixth autonomous driving device of the present invention, the driving control unit, when the evaluation unit evaluates that the validity of the reproduction start position is not met, cancels the driving in the reproduction driving mode if it is before the start of driving in the reproduction driving mode, and controls the driving unit to drive the device body back to the reproduction start position and stop at the reproduction start position if it has already started driving in the reproduction driving mode.

[0018] According to the sixth autonomous driving device of the present invention, the amount of data of the environmental image to be stored can be reduced, and there is the convenience of making it easier for the operator to perform operations such as re-selecting the work driving map when an error occurs.

[0019] To solve the above problems, the first validity evaluation method of the present invention is a validity evaluation method for evaluating the validity of the reproduction start position in a reproduction run of an autonomous driving device capable of driving based on manual operation input and autonomous driving based on automatic operation input, comprising: a measurement step of measuring the distance and angle between the device body of the autonomous driving device and objects around the device body; a map creation step of creating a local map of the area around the device body by performing coordinate transformation on the distance and angle measured in the measurement step; and, when the driving unit of the autonomous driving device drives the device body in response to manual operation input during the execution of the teaching driving mode, the measurement step and the map creation step are executed and the local map is connected so that the driving path from the teaching start position to the teaching end position of the device body and The computer is to perform the following steps: a driving control step in which a work driving map of the surrounding area of ​​the driving path is created and stored, and when the reproduction driving mode is executed, the driving unit is controlled to drive the device body at the reproduction start position by automatic operation input according to the driving path and the work driving map; an imaging step in which an imaging unit provided on the device body captures an environmental image showing the conditions around the device body; and an evaluation step in which the computer is to perform a first matching process that compares the teaching environment image and the current environment image, and acquires the environmental image captured by the imaging step when the teaching driving mode is executed as the teaching environment image, when the reproduction driving mode is executed as the current environment image, and evaluates the validity of the reproduction start position. The evaluation step involves acquiring the local map created by the map creation step at the start of the teaching driving mode as the teaching local map, acquiring the local map created by the map creation step at the start of the reproduction driving mode as the current local map, performing a second matching process that compares the teaching local map and the current local map, and evaluating the validity of the reproduction start position by the first matching process and the second matching process. It is characterized by the following: [Effects of the Invention]

[0020] According to the present invention, the autonomous driving device and its validation method make it possible to improve the accuracy of the reproduced driving without having to pre-install initialization objects along the reproduced driving route or precisely position the device body at the starting position. [Brief explanation of the drawing]

[0021] [Figure 1]It is a schematic diagram illustrating the configuration of an autonomous traveling device according to a first embodiment of the present invention. [Figure 2] It is a schematic diagram illustrating an example of an odd-numbered floor which is a cleaning area of the autonomous traveling device according to the first embodiment of the present invention. [Figure 3] It is a schematic diagram illustrating an example of an even-numbered floor which is a cleaning area of the autonomous traveling device according to the first embodiment of the present invention. [Figure 4] It is a schematic diagram illustrating an example of a work travel map for an odd-numbered floor which is a cleaning area of the autonomous traveling device according to the first embodiment of the present invention. [Figure 5] It is a schematic diagram illustrating an example of a work travel map for an even-numbered floor which is a cleaning area of the autonomous traveling device according to the first embodiment of the present invention. [Figure 6] It is a block diagram illustrating an electrical configuration of the autonomous traveling device according to the first embodiment of the present invention. [Figure 7] It is a plan view illustrating an example of a display input unit of the autonomous traveling device according to the first embodiment of the present invention. [Figure 8] It is a plan view illustrating an example of a display input unit of the autonomous traveling device according to the first embodiment of the present invention. [Figure 9] It is a plan view illustrating an example of a measurement range of a measurement unit of the autonomous traveling device according to the first embodiment of the present invention and a local map based thereon. [Figure 10] It is a plan view illustrating an example of a taught environment image acquired by an image acquisition unit and a current environment image in the autonomous traveling device according to the first embodiment of the present invention. [Figure 11] It is a flowchart illustrating an operation example of the autonomous traveling device according to the first embodiment of the present invention. [Figure 12] It is a plan view illustrating an example of a taught environment image acquired by an image acquisition unit and a current environment image in an autonomous traveling device according to a second embodiment of the present invention. [Figure 13] It is a plan view illustrating an example of an environment image captured by a rear camera of an imaging unit in an autonomous traveling device according to another example of an embodiment of the present invention. [Figure 14]This is a plan view showing an example in which image processing is applied to an environmental image captured by a rear camera of the imaging unit in an autonomous driving device according to another embodiment of the present invention. [Modes for carrying out the invention]

[0022] Embodiments of the present invention will be described below with reference to the drawings. The following embodiments are preferred examples of the present invention and disclose various preferred techniques, but the technical scope of the present invention is not limited to these embodiments.

[0023] An autonomous driving device 1 according to a first embodiment of the present invention will now be described. Figure 1 is a schematic diagram showing the configuration of the autonomous driving device 1. The autonomous driving device 1 is a vehicle capable of driving based on manual operation input and autonomous driving based on automatic operation input, and basically operates in one of the following operating modes: manual driving mode, taught driving mode, and reproduced driving mode.

[0024] As shown in Figure 1, the autonomous driving device 1 comprises a main body 2 for housing the various components, a driving unit 3 for moving the main body 2, and an operation unit 4 for receiving manual operation input for the driving unit 3. The autonomous driving device 1 can be realized as a floor cleaning machine by including a cleaning unit 5 as a work unit to perform predetermined tasks, for example, cleaning the floor surface below the main body 2. The autonomous driving device 1 can, for example, designate all or part of an area such as a shopping mall, office, hotel, hospital, school, or factory as the cleaning area 50 (work area), and the floor surface of the cleaning area 50 as the cleaning target (work target). In Figure 1, the operator of the autonomous driving device 1 is indicated by a dashed line.

[0025] If the cleaning area 50 is divided into multiple floors, the autonomous driving device 1 will perform a teaching run for each floor to store teaching run data such as a work run map 51 (see Figures 4 and 5), and then perform the cleaning work by reproducing the run based on the work run map 51 for each floor. In hotels with multiple floors, the layout may differ from floor to floor. For example, the first floor may consist of a guest room area with guest rooms, as well as a passageway to the main building and a souvenir shop. The second floor may have a completely different layout from the other floors, consisting of banquet halls and restaurants. The third to ninth floors may each mainly consist of guest room areas with multiple guest rooms.

[0026] Figure 2 is a schematic diagram showing an example of an odd-numbered floor 50a, which is the cleaning area 50 of the autonomous driving device 1, and Figure 3 is a schematic diagram showing an example of an even-numbered floor 50b, which is the cleaning area 50 of the autonomous driving device 1. Of the 3rd to 9th floors, which are the cleaning area 50, as shown in Figure 2, vending machine spaces are provided on the odd-numbered floors 50a, while as shown in Figure 3, guest rooms may be provided instead of vending machine spaces on the even-numbered floors 50b. In this case, on the odd-numbered floors 50a (3rd, 5th, 7th, and 9th floors), a common work driving map 51a (teaching driving data) can be used to reproduce the driving, and on the even-numbered floors 50b (4th, 6th, and 8th floors), a common work driving map 51b can be used to reproduce the driving.

[0027] Therefore, the autonomous driving device 1 will prepare work route maps 51 for the 1st floor, 2nd floor, odd-numbered floors 50a (3rd, 5th, 7th, and 9th floors), and even-numbered floors 50b (4th, 6th, and 8th floors). Figure 4 is a schematic diagram showing an example of a work route map 51a for odd-numbered floors 50a, which are the cleaning area 50 of the autonomous driving device 1, and Figure 5 is a schematic diagram showing an example of a work route map 51b for even-numbered floors 50b, which are the cleaning area 50 of the autonomous driving device 1. The autonomous driving device 1 will prepare work route maps 51a as shown in Figure 4 for odd-numbered floors 50a (3rd, 5th, 7th, and 9th floors), as shown in Figure 2, and will prepare work route maps 51b as shown in Figure 5 for even-numbered floors 50b (4th, 6th, and 8th floors), as shown in Figure 3. The work travel map 51 includes positional information such as the travel route 52 used for teaching, the teaching start position 53 and teaching end position 54 on the travel route 52, and the starting angle of the forward direction at the teaching start position 53. In Figures 2 to 5, the direction of travel of the autonomous driving device 1 at the teaching start position 53 is indicated by a thick arrow.

[0028] The autonomous driving device 1 includes a measurement unit 6 that measures the distance and angle (for example, the angle of the device body 2 with respect to the forward direction) between the device body 2 and obstacles (objects) such as walls and ornaments around the device body 2, and an imaging unit 7 that captures environmental images showing the conditions around the device body 2. Furthermore, the autonomous driving device 1 includes a control unit 8 that comprehensively controls each part and various functions of the autonomous driving device 1 (driving by the driving unit 3, cleaning work by the cleaning unit 5, measurement by the measurement unit 6, imaging by the imaging unit 7, etc.), a display input unit 9 for operating and displaying various functions, and a power supply unit 10 (not shown) for controlling battery charging and supplying power to each part.

[0029] The running unit 3 is integrally provided at the bottom of the main body 2 and comprises left and right front wheels 11, which are fixed (non-steering) drive wheels that can rotate independently to the left and right, one rear wheel 12, which is a swivel auxiliary wheel, and left and right drive motors (not shown) that drive the left and right front wheels 11 respectively. The front wheels 11 are provided on both sides in the left-right direction on the front underside of the main body 2, and the rear wheel 12 is provided in the center in the left-right direction on the rear underside of the main body 2. Each drive motor is provided inside the main body 2 and is connected to the rotation axis (not shown) of each front wheel 11 via a drive gear (not shown). In the running unit 3, the main body 2 can be moved forward or backward by rotating the two front wheels 11 simultaneously with each drive motor, while the main body 2 can be turned left or right by stopping the left front wheel 11 or the right front wheel 11 while simultaneously rotating the right front wheel 11 or the left front wheel 11.

[0030] Furthermore, the running unit 3 is equipped with an abnormality detection unit that detects normal or abnormal conditions, such as a step sensor (not shown) that detects a recess of a predetermined depth (e.g., 4 cm) on the lower front side of the device body 2, and a slip sensor (not shown) that detects slippage of the front wheels 11. When it detects an abnormality that hinders normal driving, it outputs an abnormality signal to the control unit 8.

[0031] The operating unit 4 includes, for example, a handle 4a, which is located at the rear of the device body 2, in the center in the left-right direction. The handle 4a is configured to be rotatable counterclockwise (left turn) and clockwise (right turn) when viewed from above. The handle 4a is equipped with an accelerator grip (not shown), and twisting the accelerator grip of the handle 4a forward allows the device body 2 to move forward, and twisting the accelerator grip of the handle 4a backward allows the device body 2 to move backward. The handle 4a can adjust the speed of forward and backward movement of the device body 2 according to the amount of operation of the accelerator grip. Furthermore, rotating the handle 4a counterclockwise allows the device body 2 to turn left, and rotating the handle 4a clockwise allows the device body 2 to turn right.

[0032] The cleaning unit 5 is configured to clean the floor surface according to the initially set cleaning conditions, cleaning conditions set via the display input unit 9, or cleaning conditions set by automatic operation input. The cleaning unit 5 is, for example, a mechanism that cleans the floor surface using a dry cleaning method, and includes a cleaning member 13 such as a roll brush for collecting dust from the floor surface, and a suction unit 14 for sucking up the dust collected by the cleaning member 13. The cleaning unit 5 also includes a bucket (not shown) for accumulating the collected dust inside. The bucket is detachable from the main body 2 of the device, and is removed when disposing of the accumulated dust and reattached after the dust has been disposed of.

[0033] The cleaning component 13 is rotatably mounted on the underside of the device body 2 and cleans the floor surface by contacting and rotating with the floor surface. The cleaning component 13 is detachable from the device body 2, i.e., it is a replaceable component. In addition to the roll brush, the cleaning component 13 may also have side brushes or the like.

[0034] The suction unit 14 includes a suction duct (not shown) for sucking up dust from the floor surface, and a suction motor (not shown) for creating negative pressure inside the suction duct. The suction unit 14 sucks up dust through the suction duct by creating negative pressure inside the suction duct using the suction motor, and collects it in a bucket. Alternatively, the suction unit 14 may be configured to suck up dust using a suction blower.

[0035] The cleaning conditions in the cleaning unit 5 described above include the operation / stopping of the rotation of the cleaning member 13, the operation / stopping of suction by the suction unit 14, the strength of the contact pressure of the cleaning member 13 against the floor surface, and the suction strength by the suction unit 14.

[0036] Furthermore, the cleaning unit 5 includes an abnormality detection unit that detects whether the operation is normal or abnormal. This unit includes, for example, a component mounting sensor (not shown) that detects when the cleaning component 13 is not attached, and a bucket collection amount detection sensor (not shown) that can detect whether the bucket is full or not. When it detects an abnormality that would hinder normal cleaning operations, it outputs an abnormality signal to the control unit 8. In addition, the cleaning unit 5 includes a collection amount calculation unit (not shown) that calculates the amount of dust to be collected during a reproduction run based on the amount of dust collected by the bucket collection amount detection sensor during a teaching run.

[0037] The measurement unit 6 includes a movement sensor that measures the amount of physical movement of the device body 2 by the travel unit 3, and a front obstacle sensor 6a that measures the distance and angle to obstacles (objects) around the front of the device body 2. In addition to the front obstacle sensor 6a, the measurement unit 6 may also include a rear obstacle sensor (not shown) that measures the distance and angle to obstacles (objects) around the rear of the device body 2.

[0038] The movement sensor consists of, for example, a pair of encoders corresponding to each of the left and right front wheels 11, and an acceleration sensor. The forward obstacle sensor 6a is located on the front of the main body 2 of the device and consists of, for example, a laser range finder (LRF), and detects obstacles that are within a predetermined measurable height (for example, 30 cm) from the floor and within a predetermined measurable range (for example, a radius of 20 m) from the forward obstacle sensor 6a. The measurement unit 6 is connected to the control unit 8 and transmits the measurement results to the control unit 8.

[0039] Furthermore, when the autonomous driving device 1 is operating in reenactment mode, the measurement unit 6 also functions as an abnormality detection unit. When an ultrasonic sensor (not shown), which is provided separately from the forward obstacle sensor 6a, detects an obstacle within a predetermined limited range (for example, a radius of 50 cm) from the main body of the device 2, it outputs an abnormality signal to the control unit 8.

[0040] The imaging unit 7 processes environmental images captured by cameras attached to the main body 2 of the autonomous driving device 1. The imaging unit 7 has at least a front camera 7a capable of capturing the foreground of the main body 2 including the floor, and a rear camera 7b capable of capturing the background of the main body 2 including the floor.

[0041] The imaging unit 7 is capable of continuously acquiring environmental images using the front camera 7a and the rear camera 7b. The front camera 7a and the rear camera 7b are composed of, for example, a CCD (Charge Coupled Device) camera or a TOF (Time of Flight) camera. The imaging unit 7 may also have a side camera (not shown) that captures images to the side (left or right) of the main body 2. In Figure 1, for simplification, an example is shown in which the front camera 7a and the rear camera 7b are integrally mounted on the top of the main body 2, but the front camera 7a and the rear camera 7b may be mounted at separate locations on the main body 2.

[0042] The imaging unit 7 can capture still images and store them in the storage unit 16, and can also capture and store moving images in the storage unit 16. Furthermore, the imaging unit 7 can output the captured environmental images to the display input unit 9 for real-time display. The front camera 7a and rear camera 7b may be equipped with lenses of generally standard field-of-view, but may also be equipped with wide-angle lenses. Alternatively, only the rear camera 7b may be equipped with a relatively wide-angle lens.

[0043] The control unit 8 consists of a CPU (Central Processing Unit) and the like. Figure 6 is a block diagram showing the electrical configuration of the autonomous driving device 1. As shown in Figure 6, the control unit 8 is connected to the storage unit 16 via a bus 17, and the bus 17 is further connected to an interface 18. The control unit 8 is also connected to the driving unit 3, operation unit 4, cleaning unit 5, measurement unit 6, imaging unit 7, display input unit 9, and power supply unit 10 via the bus 17 and interface 18.

[0044] The memory unit 16 consists of ROM (Read Only Memory), RAM (Random Access Memory), hard disk, flash memory, etc., and stores programs and data for controlling each part and various functions of the autonomous driving device 1. The control unit 8 reads the programs and data stored in the memory unit 16 and controls each part and various functions by executing these programs. The control unit 8 may also be equipped with a dedicated controller, a GPU (Graphics Processing Unit), which is particularly specialized for calculation processing related to real-time image processing.

[0045] Furthermore, the control unit 8 includes a mode switching unit 20, a map creation unit 21, a teaching driving control unit 22, a reproduction driving control unit 23, an image acquisition unit 24, a validity evaluation unit 25, and an evaluation result output unit 26. The mode switching unit 20, the map creation unit 21, the teaching driving control unit 22, the reproduction driving control unit 23, the image acquisition unit 24, the validity evaluation unit 25, and the evaluation result output unit 26 may be composed of programs stored in the storage unit 16 and executed by the control unit 8. For example, in the autonomous driving device 1, a validity evaluation processing program that evaluates the validity of the starting position (reproduction start position) of the reproduction driving of the autonomous driving device 1 is stored in the storage unit 16. This validity evaluation processing program causes a computer such as the control unit 8 to execute the mode switching unit 20, the map creation unit 21, the teaching driving control unit 22, the reproduction driving control unit 23, the image acquisition unit 24, the validity evaluation unit 25, and the evaluation result output unit 26. The individual components of the control unit 8 will be described later.

[0046] Figures 7(a), 7(b), and 8 are plan views showing an example of the display input unit 9 of the autonomous driving device 1. The display input unit 9 is located on the upper surface of the device body 2, in front of the handle 4a, and includes a main switch 30, a forced stop button 31, a speaker 32, and a display 33, as shown in Figures 7(a), 7(b), and 8. Each part of the display input unit 9 is connected to the control unit 8.

[0047] The main switch 30 is configured to be switchable between "off," "manual," and "automatic." Switching the main switch 30 to "manual" switches the operating mode of the autonomous driving device 1 to manual driving mode via the mode switching unit 20, while switching the main switch 30 to "automatic" switches the operating mode of the autonomous driving device 1 to simulated driving mode via the mode switching unit 20.

[0048] The forced stop button 31 operates to forcibly stop the autonomous driving device 1, and outputs a forced stop signal to the control unit 8 in response to the operation. The speaker 32 generates an alarm sound or the like in response to the control unit 8's control.

[0049] The display unit 33 consists of a touch panel and displays various screens in response to control signals from the control unit 8, and transmits operation signals to the control unit 8 based on touch operations on each screen. The display unit 33 displays the operating status (operating mode) and operating time of the autonomous driving device 1, various setting items in the teaching driving mode and reproduction driving mode, information regarding the validity of the starting position (reproduction start position) in the reproduction driving mode, cleaning conditions of the cleaning unit 5, warnings and error codes, etc., in response to control signals from the control unit 8.

[0050] For example, the display unit 33 displays the learning key 40, the cleaning operation key 41, the directional keys 42, the cleaning condition setting key 43, and the display area 44 in an operable manner. In this embodiment, an example is described in which the learning key 40, the cleaning operation key 41, the directional keys 42, and the cleaning condition setting key 43 are operablely displayed on the display unit 33. However, at least one of the learning key 40, the cleaning operation key 41, the directional keys 42, and the cleaning condition setting key 43 may be provided separately in the display input unit 9 from the display unit 33.

[0051] The learning key 40 is configured to be operable when the main switch 30 is set to "manual". By operating the learning key 40 (for example, by pressing and holding it), the mode switching unit 20 switches the operating mode of the autonomous driving device 1 to the teaching driving mode, and the execution of the teaching driving mode begins. By operating the learning key 40 again, the execution of the teaching driving mode ends, and the mode switching unit 20 switches the operating mode of the autonomous driving device 1 back to manual driving mode. The learning key 40 also functions as a recording switch to record the mark positions 55, 56, and 57 (see Figures 2 and 3) during teaching driving, as will be described later.

[0052] The cleaning operation key 41 is configured to be switchable between "on" and "off". Switching the cleaning operation key 41 to "on" causes the cleaning unit 5 to operate according to the cleaning conditions.

[0053] The directional pad 42 accepts selection operations for various setting items displayed on the display unit 33 and transmits signals related to those selection operations to the control unit 8.

[0054] The cleaning condition setting key 43 receives setting operations for cleaning conditions, such as switching the operation of each part of the cleaning unit 5, and transmits signals related to these setting operations to the control unit 8. The cleaning condition setting key 43 includes, for example, a ground pressure key 43a, a rotation key 43b, and a suction key 43c.

[0055] The ground pressure key 43a switches the intensity of the ground pressure of the cleaning member 13 (roll brush) against the floor surface to one of several levels, and the intensity is cyclically switched with each operation. The rotation key 43b switches the operation and stop of the rotation of the cleaning member 13, and switches between "on" and "off" with each operation. The suction key 43c switches the operation and stop of the suction by the suction unit 14, and switches between "on" and "off" with each operation. The cleaning condition setting key 43 may also have a suction strength key (not shown) that switches the suction strength of the suction unit 14 to one of several levels.

[0056] As shown in Figure 7(a), the display area 44 functions as an image display area that displays the current environmental image captured by the front camera 7a or rear camera 7b of the imaging unit 7, for example, when the manual driving mode or the teaching driving mode is being executed. The display area 44 may also function as an image display area when the reproduction driving mode is being executed. The display unit 33 can switch between a normal display mode (see Figure 7(a)) in which the image display area is displayed on a portion of the screen of the display unit 33 in response to a touch operation of the display area 44, and the learning key 40, cleaning operation key 41, cross key 42, and cleaning condition setting key 43 are made operable, and a full-screen mode (see Figure 7(b)) in which the image display area is displayed on the entire screen of the display unit 33.

[0057] Furthermore, as shown in Figure 8, the display area 44 functions as a map selection area for selecting a work route map 51 to be used for teaching or reproduction driving at the start of teaching or reproduction driving mode. As a map selection area, the display area 44 displays a list of work route maps 51 already stored in the storage unit 16, which can be selected from the map list 45. In reproduction driving mode, the map selection area displays an operable execution key 46 for executing the reproduction driving of the work route map 51 selected from the map list 45, and also displays the evaluation result 47 of the validity of the work route map 51 by the validity evaluation unit 25, which will be described later. In addition, the map selection area may also display an operable back key 48 for returning the map list 45 to an unselected state and re-selecting a work route map 51 after selecting a work route map 51, and an automatic selection key 49 for automatically selecting a work route map 51 with a high evaluation result.

[0058] For example, as shown in Figure 8, the map list 45 in the display area 44, which serves as the map selection area, displays a list of map names so that the following can be selected: the work route map 51 for the 1st floor, the work route map 51 for the 2nd floor, the work route map 51a for the odd-numbered floors 50a (3rd, 5th, 7th, and 9th floors), and the work route map 51b for the even-numbered floors 50b (4th, 6th, and 8th floors). The display area 44 may display the map name of the work route map 51 selected in the map selection area and the corresponding evaluation result display 47 in a different display format from other map names and evaluation result displays 47, for example, by displaying them in white text.

[0059] The power supply unit 10 is equipped with a battery (power supply) mounted inside the main unit 2 and is configured to be rechargeable by connecting to an external power supply. The power supply unit 10 includes a power supply remaining capacity detection unit that detects the remaining capacity of the battery, and a power supply capacity calculation unit that calculates the power (power supply capacity) required for the reproduction run based on the power consumed during the teaching run. The power supply unit 10 then determines whether or not to perform the reproduction run by comparing the remaining battery capacity detected by the power supply remaining capacity detection unit with the power supply capacity calculated by the power supply capacity calculation unit. For example, if the power supply unit 10 determines that the reproduction run cannot be completed when the remaining battery capacity is less than the power supply capacity required for the reproduction run. If it determines that the reproduction run is possible, it proceeds to the reproduction run; if it determines that the reproduction run is impossible, it outputs a signal to the control unit 8 indicating that the reproduction run is impossible and displays an error.

[0060] Next, we will describe the various components of the control unit 8.

[0061] The mode switching unit 20 switches the operating mode of the autonomous driving device 1 to manual driving mode, teaching driving mode, or reproduction driving mode in response to the operation of the main switch 30 and learning key 40 of the display input unit 9. While the operating mode is set to manual driving mode or teaching driving mode by the mode switching unit 20, manual operation input of the operation unit 4 becomes possible. The teaching driving mode is a mode in which, in manual driving mode, the autonomous driving device 1 is programmed to record the operator's operation input in the cleaning area 50 in response to the operator's memorized operations, and a work driving map 51 linked to the work information is created. The reproduction driving mode is a mode in which the autonomous driving device 1 is programmed to autonomously drive according to the driving route 52 and linked work information based on the work driving map 51 to be executed in automatic driving mode.

[0062] The map creation unit 21 uses technologies such as SLAM (Simultaneous Localization and Mapping) to estimate its own position and create an environmental map in real time. The map creation unit 21 creates a local map of the area around the device body 2 at a predetermined location by transforming the coordinates of the distance and angle from obstacles (objects) measured by the measurement unit 6, and estimates the self-position of the device body 2 on the local map based on the created local map and the amount of movement of the device body 2 measured by the measurement unit 6.

[0063] Figure 9(a) is a plan view showing an example of the measurement range of the measurement unit 6 of the autonomous driving device 1, and Figure 9(b) is a plan view showing an example of a local map based on this measurement range. For example, as shown in Figure 9(a), the measurement unit 6 has a detection range that is approximately 180 degrees in a fan shape in front of the device body 2. In this case, if the autonomous driving device 1 is located in a passage with walls on both sides as shown in Figures 2 and 3, the map creation unit 21 will create a local map that can determine the distance and shape of the left and right walls relative to the autonomous driving device 1, as shown in Figure 9(b).

[0064] The map creation unit 21 stores the local map created during the execution of the teaching driving mode in the storage unit 16 as the teaching local map. For example, when the execution of the teaching driving mode is started, the map creation unit 21 creates the teaching local map while the autonomous driving device 1 is positioned at the teaching start position (teaching start position 53). Specifically, the map creation unit 21 determines that the teaching driving mode is to be started when the learning key 40 is pressed in manual driving mode to start the teaching driving mode, or when forward driving is started or the cleaning unit 5 is started in teaching driving mode. When the autonomous driving device 1 is positioned at the teaching start position 53 in Figures 2 and 3 and the teaching driving mode is started, the map creation unit 21 generates a local map showing the surrounding environment conditions measured by the measurement unit 6 at the start of the teaching driving mode using planar image data.

[0065] Furthermore, the map creation unit 21 stores the local map created during the execution of the reenactment driving mode as the current local map in the storage unit 16. For example, when the execution of the reenactment driving mode is started, the map creation unit 21 creates the current local map while the autonomous driving device 1 is positioned at the reenactment driving start position (reenactment start position). Specifically, the map creation unit 21 determines that the execution of the reenactment driving mode should begin when the main switch 30 is switched to "automatic" and the system switches to the reenactment driving mode, when the work driving map 51 is selected in the display area 44 as the map selection area of ​​the display unit 33 in the reenactment driving mode and the reenactment driving mode is started, or when the system starts moving forward in the reenactment driving mode. When the autonomous driving device 1 moves toward the teaching start position 53 in Figures 2 and 3 and then starts the execution of the reenactment driving mode, the map creation unit 21 generates a local map showing the surrounding environment conditions measured by the measurement unit 6 at the start of the execution of the reenactment driving mode using planar image data.

[0066] The teaching and driving control unit 22, in a predetermined cleaning area 50 (work area) (see Figures 2 and 3), when executing the teaching and driving mode, creates a work driving map 51 (see Figures 4 and 5) around the driving path 52 while the driving unit 3 drives the device body 2 in response to manual input from the operation unit 4, and stores it in the storage unit 16. Specifically, the teaching and driving control unit 22 operates the measurement unit 6 and the map creation unit 21 to create local maps at each position on the driving path 52 of the device body 2, and further synthesizes the local maps of each position to create the work driving map 51. In addition to map data, the work driving map 51 also includes position information obtained based on the self-position estimation of the device body 2 (such as the driving path 52, the teaching start position 53 and teaching end position 54 on the driving path 52, and the starting angle of the forward direction at the teaching start position 53), as well as information on the amount of movement of the device body 2 at each position on the driving path 52 (such as speed and turning angle).

[0067] Furthermore, when the teaching driving mode is executed, the teaching driving control unit 22 stores the cleaning conditions of the cleaning unit 5, which have been set according to the manual operation input of the cleaning condition setting key 43 of the display input unit 9, in the storage unit 16 in correspondence with the position on the driving path 52 of the work driving map 51.

[0068] The teaching and driving control unit 22 can store multiple work driving maps 51 in the storage unit 16, and can not only create new work driving maps 51 but also edit work driving maps 51 that have already been stored.

[0069] When creating a new work travel map 51, the teaching travel control unit 22 displays an instruction to input the map name of the work travel map 51 on the display input unit 9's display 33 when the teaching travel mode is started. Then, when the map name is entered using the display 33 or the directional keys 42, the teaching travel control unit 22 starts executing the teaching travel mode to create the work travel map 51 for that map name.

[0070] When editing a work driving map 51, the teaching driving control unit 22 displays a map selection area in the display area 44 of the display unit 33 of the display input unit 9 when the teaching driving mode is started, and displays the work driving maps 51 already stored in the storage unit 16 in a selectable list in the map list 45. Then, when a map name is selected by selection operation using the directional keys 42 or direct touch operation on the display unit 33, the teaching driving control unit 22 starts executing the teaching driving mode to create a work driving map 51 for that map name.

[0071] The teaching and driving control unit 22 may recreate (replace) the work driving map 51 by combining each local map at the end of the teaching and driving mode after the device body 2 has traveled to the teaching end position 54, or it may create the work driving map 51 in real time (partially stitched together) by combining each local map created up to that point in parallel with the movement of the device body 2.

[0072] When the reproduction driving mode is executed, the reproduction driving control unit 23 reads the work driving map 51 to be executed from the storage unit 16, and automatically inputs operation commands to the driving unit 3 based on this work driving map 51, and controls the driving unit 3 so that the main unit 2 performs a reproduction drive according to the driving path 52 of the work driving map 51.

[0073] Furthermore, when the reproduction driving mode is executed, the reproduction driving control unit 23 reads cleaning conditions corresponding to the position of the device body 2 on the driving path 52 from the storage unit 16 while the driving unit 3 is making the device body 2 perform a reproduction drive according to the driving path 52 of the work driving map 51, and controls the cleaning unit 5 to perform the cleaning work by automatic operation input according to these cleaning conditions.

[0074] Specifically, the replay driving control unit 23 displays a map selection area in the display area 44 of the display input unit 9's display unit 33 to allow the user to select a work driving map 51 from among several work driving maps 51 on which to execute the replay driving mode. The work driving maps 51 already stored in the storage unit 16 are displayed in a selectable list in the map list 45. When a map name is selected using the directional keys 42 or by direct touch operation on the display unit 33, and the execution key 46 is pressed, the replay driving control unit 23 starts executing the replay driving mode according to the work driving map 51 of that name.

[0075] When preparing the autonomous driving device 1 for a simulated run, the operator moves the autonomous driving device 1 from a storage location such as a warehouse towards the teaching start position 53 of the work run map 51 to be simulated. The position where the autonomous driving device 1 is positioned after the move is completed becomes the position where the simulated run actually begins (simulation start position). Here, the validity evaluation unit 25 evaluates the validity of the simulation start position in relation to the teaching start position 53 of the work run map 51, as described below. If the validity is evaluated as low or not meeting the validity requirements, the simulation run control unit 23 will disable the simulation run of the work run map 51. For example, before starting a run in simulation mode, the simulation run control unit 23 will cancel the simulation run of the selected work run map 51.

[0076] The image acquisition unit 24 acquires environmental images captured by the imaging unit 7 according to predetermined acquisition timings.

[0077] For example, when the teaching driving mode is executed, the image acquisition unit 24 acquires an environmental image captured by the imaging unit 7 and stores it in the storage unit 16 as a teaching environment image 60 (see Figure 10(a)). Specifically, the image acquisition unit 24 acquires an environmental image captured by the imaging unit 7 (especially the front camera 7a) as a teaching environment image 60 (start position image) when the autonomous driving device 1 is positioned at the teaching start position 53, using the start of teaching driving, such as when the learning key 40 is pressed in manual driving mode to start teaching driving mode, or when forward driving starts or the cleaning unit 5 starts operating, as the acquisition timing. Furthermore, when the teaching driving control unit 22 ends teaching driving mode and creates a work driving map 51, the image acquisition unit 24 stores the teaching environment image 60 in association with the work driving map 51 (map name).

[0078] Figure 10(a) is a plan view showing an example of a teaching environment image 60 acquired by the image acquisition unit 24 in the autonomous driving device 1. When the autonomous driving device 1 performs a teaching run on the third floor as shown in Figure 2, the imaging unit 7 captures an environment image as shown in Figure 10(a) when the autonomous driving device 1 is positioned at the teaching start position 53, and the image acquisition unit 24 stores the environment image captured by the imaging unit 7 as a teaching environment image 60 in the storage unit 16, associating it with the work run map 51 of the third floor.

[0079] Furthermore, when the reenactment driving mode is executed, the image acquisition unit 24 acquires the environmental image captured by the imaging unit 7 and stores it in the storage unit 16 as the current environmental image 61 (see Figure 10(b)). Specifically, the image acquisition unit 24 acquires the environmental image captured by the imaging unit 7 (especially the front camera 7a) as the current environmental image 61 when the autonomous driving device 1 is positioned at the reenactment start position, using the start of the reenactment driving mode as the acquisition timing, such as when the main switch 30 is switched to "automatic" and the reenactment driving mode is switched on, when the work driving map 51 is selected in the display area 44 as the map selection area of ​​the display unit 33 in the reenactment driving mode and the reenactment driving mode is started, or when forward driving is started in the reenactment driving mode.

[0080] Figure 10(b) is a plan view showing an example of a current environmental image 61 acquired by the image acquisition unit 24 in the autonomous driving device 1. When the autonomous driving device 1 performs a recreation run on the 4th floor as shown in Figure 3, the imaging unit 7 captures an environmental image as shown in Figure 10(b) when the autonomous driving device 1 is positioned at the recreation start position, and the image acquisition unit 24 stores the environmental image captured by the imaging unit 7 as the current environmental image 61 in the storage unit 16. If the work run map 51 on the 3rd floor is selected for the recreation run, the current environmental image 61 at the recreation start position on the 4th floor will have differences 62 from the teaching environmental image 60 at the teaching start position 53 on the 3rd floor, as shown in Figure 10(b).

[0081] The validation unit 25 evaluates the validity of the reproduction start position to determine whether the starting position (reproduction start position) at which the autonomous driving device 1 starts the reproduction drive corresponds to the teaching start position 53 of the driving path 52 included in the work driving map 51 on which the reproduction drive is performed, according to a predetermined evaluation timing.

[0082] At this time, the validation evaluation unit 25 performs a matching process (first matching process) by comparing the teaching environment image 60 acquired by the image acquisition unit 24 when the teaching driving mode is executed with the current environment image 61 acquired by the image acquisition unit 24 when the reproduction driving mode is executed, and evaluates the validity of the reproduction start position. Specifically, the validation evaluation unit 25 evaluates the validity of the reproduction start position by performing the first matching process at the start of the reproduction driving, such as when the main switch 30 is switched to "automatic" and the system switches to the reproduction driving mode, or when the work driving map 51 is selected from the map list 45 in the display area 44 as the map selection area of ​​the display unit 33 in the reproduction driving mode and the reproduction driving mode is started, or when forward driving is started in the reproduction driving mode.

[0083] The validation unit 25 calculates a matching rate (matching score) indicating the degree to which the current environmental image 61 matches the teaching environmental image 60 by performing a first matching process between the teaching environmental image 60 and the current environmental image 61 associated with the selected work travel map 51. The validation unit 25 calculates the matching rate between the teaching environmental image 60 and the current environmental image 61 obtained by the first matching process as a percentage, and may calculate it as a maximum of 100%.

[0084] The validity evaluation unit 25 evaluates the validity as high or satisfactory if the agreement rate exceeds a predetermined agreement rate threshold (for example, 50%), and evaluates the validity as low or unsatisfactory if the agreement rate is below the predetermined agreement rate threshold. Alternatively, the validity evaluation unit 25 may evaluate the degree to which the agreement rate satisfies the validity requirement in multiple stages by comparing the agreement rate with multiple agreement rate thresholds.

[0085] The validation unit 25 may, for example, perform pattern matching as a first matching process, comparing a portion of the teaching environment image 60 with comparison images extracted at each position of the current environment image 61 at the same size as the portion of the image, calculating the agreement rate for each comparison image at each position, and selecting the comparison image with the highest agreement rate as the candidate comparison image. The validation unit 25 extracts an image of the central part of the teaching environment image 60 as a portion of the teaching environment image 60, for example, as shown by the dashed line in Figure 10(b).

[0086] The validation evaluation unit 25 performs pattern matching, which allows it to calculate the agreement rate of the first matching process with little deviation from the normal state, even if the orientation of the teaching environment image 60 and the current environment image 61 are different or shifted in the left-right direction. Therefore, when the operator moves the autonomous driving device 1 to the reproduction start position of the reproduced driving, they can focus on the position itself and obtain an appropriate agreement rate of the first matching process without having to precisely position the direction of travel (angle) relative to the teaching start position 53, thereby simplifying the operator's work.

[0087] Furthermore, the validation evaluation unit 25 performs coordinate transformation processing and distortion correction processing on the teaching environment image 60 and the current environment image 61 as needed. In addition, the validation evaluation unit 25 performs semantic segmentation processing on the teaching environment image 60 and the current environment image 61, classifying each image into segmented images according to attributes such as floor, wall, ceiling, and door. By performing coordinate transformation processing, distortion correction processing and semantic segmentation processing, the validation evaluation unit 25 facilitates the calculation of the agreement rate in the first matching process of the teaching environment image 60 and the current environment image 61, thereby increasing the processing speed and reducing the capacity used by the storage unit 16, and enabling accurate validation evaluation.

[0088] The validation unit 25 can perform a first matching process on the teaching environment image 60 and the current environment image 61 while suppressing the effects of image brightness and lighting at the time of acquisition through semantic segmentation processing. The validation unit 25 may also remove elements other than the outline from each segmented image classified in the teaching environment image 60 and the current environment image 61 through semantic segmentation processing and apply them to the first matching process. For example, the pattern of the carpet on the floor, the pattern of the wallpaper on the wall, the lighting on the ceiling, etc., may be removed from the teaching environment image 60 and the current environment image 61. Furthermore, the validation unit 25 may also remove people, small animals, etc., from the teaching environment image 60 and the current environment image 61 through semantic segmentation processing, thereby performing the first matching process using only the surrounding image and improving the reliability of the processing results. Note that the image after semantic segmentation processing becomes an image with several different color divisions, as shown in Figure 14, and the detailed patterns and shading of the original image are removed to some extent.

[0089] The validation unit 25 may perform either the process of extracting a portion of the teaching environment image 60 or the semantic segmentation process of the teaching environment image 60 and the current environment image 61 in any order.

[0090] Incidentally, the validation evaluation unit 25 may compare a portion of the teaching environment image 60 with comparison images of each position in the current environment image 61 and detect at least one second candidate comparison image that has obtained a similar agreement rate to the first candidate comparison image, in addition to the first candidate comparison image that has obtained the highest agreement rate. If multiple candidate comparison images are detected in this way, the validation evaluation unit 25 may recalculate the agreement rate of the first candidate comparison image and use that as the evaluation result.

[0091] In this case, if the positions of the first candidate comparison image and the second candidate comparison image are not separated by more than a predetermined distance threshold, the validity evaluation unit 25 determines that the second candidate comparison image is within the range of error (it is a slightly shifted image) relative to the first candidate comparison image and adopts the agreement rate of the first candidate comparison image as is. On the other hand, if the positions are separated by more than a predetermined distance threshold, the unit determines that the second candidate comparison image is not within the range of error relative to the first candidate comparison image, evaluates the first candidate comparison image as being in an ambiguous state, recalculates the agreement rate of the first candidate comparison image to a lower value and adopts it.

[0092] Furthermore, if the difference in the agreement rates between the first candidate comparison image and the second candidate comparison image is not less than a predetermined difference threshold, the validity evaluation unit 25 determines that the agreement rate of the second candidate comparison image is not approximate to the first candidate comparison image and adopts the agreement rate of the first candidate comparison image as is. On the other hand, if the difference in the agreement rates between the first candidate comparison image and the second candidate comparison image is less than a predetermined difference threshold, the validity evaluation unit 25 determines that the agreement rate of the second candidate comparison image is approximate to the first candidate comparison image, evaluates the first candidate comparison image as being in an ambiguous state, recalculates the agreement rate of the first candidate comparison image to a lower value and adopts it.

[0093] The validity evaluation unit 25 may recalculate the agreement rate of the first candidate comparison image to a lower value if either the positions of the first candidate comparison image and the second candidate comparison image are separated by a predetermined distance threshold or the difference in the agreement rate of the first candidate comparison image and the second candidate comparison image is less than a predetermined difference threshold, or if both conditions are met. The validity evaluation unit 25 calculates a lower agreement rate, for example, by performing a predetermined subtraction on the highest agreement rate of the first candidate comparison image.

[0094] The validation unit 25 may create several types of teaching environment images 60 by enlarging or reducing the teaching environment image 60 by a predetermined magnification or by rotating it by a predetermined angle, and perform a first matching process between each teaching environment image 60 and the current environment image 61. Alternatively, the validation unit 25 may create several types of current environment images 61 by enlarging or reducing the current environment image 61 by a predetermined magnification or by rotating it by a predetermined angle, and perform a first matching process between each teaching environment image 60 and the current environment image 61. This allows the operator to obtain an appropriate agreement rate in the first matching process without having to position the direction (angle) of travel with high precision, thereby simplifying the operator's work.

[0095] Furthermore, the validation evaluation unit 25 may, in addition to the first matching process, perform a matching process (second matching process) in which it compares the teaching-time local map acquired by the map creation unit 21 during the execution of the teaching driving mode with the current local map acquired by the map creation unit 21 during the execution of the reproduction driving mode, and evaluate the validity of the reproduction start position based on the processing results of the first matching process and the processing results of the second matching process. The validation evaluation unit 25 may be configured to allow operation to set whether or not to perform the second matching process on or off.

[0096] The validation unit 25 calculates a matching rate (matching score) indicating the degree to which the current local map matches the local map at the time of teaching by performing a second matching process between the local map at the time of teaching and the current local map. The validation unit 25 calculates the matching rate between the local map at the time of teaching and the current local map obtained from the second matching process as a percentage, and may calculate it as a maximum of 100%.

[0097] The validation evaluation unit 25 performs a second matching process between the teaching-time local map and the current local map associated with the selected work travel map 51, and may, for example, perform pattern matching. Both the teaching-time local map and the current local map are so-called monochrome (filled) image data acquired as images from the map creation unit 21. In the pattern matching of the second matching process, if the validation evaluation unit 25 detects multiple candidate comparison images when comparing the image acquired as the teaching-time local map with multiple comparison images acquired at each position on the current local map, it may recalculate the agreement rate of the first candidate comparison image that obtained the highest agreement rate, in the same manner as in the first matching process, and use this as the evaluation result.

[0098] The validation unit 25 may create several types of local maps at the time of teaching by rotating the local map at the time of teaching by a predetermined angle, and perform a second matching process between each local map at the time of teaching and the current local map. Alternatively, the validation unit 25 may create several types of current local maps by rotating the current local map by a predetermined angle, and perform a second matching process between each local map at the time of teaching and the current local map.

[0099] When performing the first matching process and the second matching process, the validation evaluation unit 25 comprehensively evaluates the validity of the reproduction start position by combining the processing results of the first matching process and the processing results of the second matching process. For example, the validation evaluation unit 25 calculates the overall agreement rate by calculating the product of the first agreement rate of the processing results of the first matching process and the second agreement rate of the processing results of the second matching process. Alternatively, the validation evaluation unit 25 may calculate the overall agreement rate by calculating the sum of the first agreement rate and the second agreement rate, or it may calculate the overall agreement rate using another calculation formula that uses the first agreement rate and the second agreement rate.

[0100] Furthermore, the validation evaluation unit 25 may not only evaluate the validity of the reproduction start position based on the processing results of the first matching process and the overall processing results of the first and second matching processes, but may also evaluate the validity of the work travel map 51 to determine whether the work travel map 51 selected for reproduction travel corresponds to the cleaning area 50 where the autonomous travel device 1 is located.

[0101] The evaluation result output unit 26 outputs the validity evaluation result of the reproduction start position by the validity evaluation unit 25 in a format and by means of the display input unit 9 that is easy for the operator to understand. For example, the evaluation result output unit 26 outputs a first evaluation score as the validity evaluation result, based on the agreement rate (matching rate) between the teaching environment image 60 and the current environment image 61 obtained by the first matching process.

[0102] At this time, the evaluation result output unit 26 outputs the numerical information indicating the first evaluation score, using the numerical value of the agreement rate of the first matching process as the first evaluation score. Specifically, if the agreement rate of the first matching process is 52%, the evaluation result output unit 26 displays the numerical information "52" using the numerical part "52" as the first evaluation score, and if the agreement rate of the first matching process is 94%, it displays the numerical information "94" using the numerical part "94" as the first evaluation score, and so on, displaying the first evaluation score out of a possible 100 points.

[0103] Furthermore, the evaluation result output unit 26 may identify the first evaluation score in stages and change the display color of the numerical information accordingly. For example, if the first evaluation score is below a predetermined score threshold (e.g., 50 points), the evaluation result output unit 26 may determine that the validity of the reproduction start position is low and notify the operator of the low validity by changing the display color of the numerical information to inverted characters or outputting an alarm. The evaluation result output unit 26 may also allow the notification of low validity to be turned on or off according to the operation. In addition, the evaluation result output unit 26 may output the numerical information by displaying a graph showing the first evaluation score, etc.

[0104] As shown in Figure 8, when the evaluation result output unit 26 displays a map selection area for selecting a work run map 51 to be reproduced in the display area 44 of the display unit 33 of the display input unit 9, it displays the first evaluation score in the evaluation result display 47 corresponding to the work run map 51 that underwent validity evaluation from among the work run maps 51 that are listed in the map list 45 for selection. Alternatively, the evaluation result output unit 26 may display the first evaluation score in an area other than the evaluation result display 47 of the map list 45, or it may display the first evaluation score using a pop-up display or other display format.

[0105] Furthermore, if the validation evaluation unit 25 performs a second matching process in addition to the first matching process, the evaluation result output unit 26 outputs the validation result of the reproduction start position, which is a combination of the processing results of the first matching process and the processing results of the second matching process, via the display input unit 9. For example, the evaluation result output unit 26 outputs a total evaluation score as numerical information, which is a combination of a first evaluation score based on the agreement rate (matching rate) between the teaching environment image 60 and the current environment image 61 obtained by the first matching process, and a second evaluation score based on the agreement rate (matching rate) between the teaching local map and the current local map obtained by the second matching process.

[0106] Specifically, the evaluation result output unit 26 obtains the numerical value of the agreement rate of the second matching process as the second evaluation score, in the same manner as the first evaluation score, and displays the overall evaluation score, which is calculated by combining the first evaluation score and the second evaluation score by multiplication or summation, on the evaluation result display 47 corresponding to the work route map 51 on which the validity evaluation was performed. Alternatively, the evaluation result output unit 26 displays the numerical value of the overall agreement rate calculated by the validity evaluation unit 25 based on the first agreement rate of the first matching process and the second agreement rate of the second matching process, as the overall evaluation score on the evaluation result display 47 corresponding to the work route map 51 on which the validity evaluation was performed.

[0107] Furthermore, if the validation evaluation unit 25 detects a difference 62 between the teaching environment image 60 and the current environment image 61 as a result of the first matching process, the evaluation result output unit 26 may display the display area 44 of the image display area on the display unit 33 of the display input unit 9, display the current environment image 61 in the display area 44, and indicate the difference 62 in the corresponding location of the current environment image 61.

[0108] Figure 11 is a flowchart showing an example of the operation of the autonomous driving device 1. Next, an example of the operation of the autonomous driving device 1 will be explained with reference to the flowchart in Figure 11.

[0109] First, the autonomous driving device 1 is activated when the operator operates the main switch 30 on the display input unit 9. If the main switch 30 is switched to "manual" (step S1: YES), the operating mode of the autonomous driving device 1 is switched to manual driving mode by the mode switching unit 20 of the control unit 8 (step S2).

[0110] In manual driving mode, the autonomous driving device 1 can be driven by manual operation input using the handle part 4a. For example, the operator can manually move the autonomous driving device 1 from a warehouse or the like to a cleaning area 50 (work area) (step S3).

[0111] Next, when the operator operates the learning key 40 on the display input unit 9 to create the work driving map 51 (Step S4: YES), the operating mode of the autonomous driving device 1 is switched to the teaching driving mode by the mode switching unit 20 of the control unit 8 (Step S5).

[0112] In the teaching driving mode, it becomes possible to perform teaching driving related to the driving of the autonomous driving device 1 in response to manual operation of the handle unit 4a, and floor cleaning work in response to operation of the cleaning operation key 41 and cleaning condition setting key 43 on the display input unit 9 (step S6).

[0113] Furthermore, the image acquisition unit 24 acquires an environmental image captured by the imaging unit 7 (forward camera 7a) as a teaching environment image 60 (start position image) with the autonomous driving device 1 positioned at the teaching start position 53, using the start of the teaching run as the acquisition timing, and stores it in the storage unit 16 in association with the work run map 51 (step S7). In addition, the map creation unit 21 creates a teaching local map with the autonomous driving device 1 positioned at the teaching start position 53, and stores it in the storage unit 16 in association with the work run map 51.

[0114] While the teaching run is being performed, the teaching run control unit 22 activates the measurement unit 6 and the map creation unit 21 to create a work run map 51 along the run path 52 and store it in the storage unit 16 (step S8).

[0115] On the other hand, if the main switch 30 is not set to "manual" (Step S1: NO) or if the learning key 40 is not operated (Step S4: NO), and the main switch 30 is switched to "automatic" (Step S9: YES), the reproduction driving control unit 23 checks for the existence of the work driving map 51. If the work driving map 51 does not exist, an error is displayed on the display input unit 9 to prompt the operator to take action. If the work driving map 51 exists, the operating mode of the autonomous driving device 1 is switched to the reproduction driving mode by the mode switching unit 20 of the control unit 8 (Step S10).

[0116] In the replay driving mode, first, a display area 44, which serves as a map selection area, is displayed on the display input unit 9's display unit 33, showing a map list 45 of the work driving maps 51 stored in the memory unit 16. The user then operates the directional keys 42 to select the work driving map 51 to be executed (step S11). For example, in this example, the work driving map 51b for an even-numbered floor 50b is selected.

[0117] When the work travel map 51 is selected, the image acquisition unit 24 acquires the environmental image captured by the imaging unit 7 (forward camera 7a) as the current environmental image 61 with the autonomous driving device 1 positioned at the reproduction start position (step S12). Furthermore, the map creation unit 21 creates a current local map with the autonomous driving device 1 positioned at the reproduction start position. For example, in this example, it is assumed that the autonomous driving device 1 has moved toward the teaching start position 53 on an odd-numbered floor 50a and is positioned at the reproduction start position.

[0118] Furthermore, the validity evaluation unit 25 performs a first matching process that compares the teaching environment image 60 with the current environment image 61, and a second matching process that compares the teaching local map with the current local map to evaluate the validity of the reproduction start position (step S13). For example, in this example, the autonomous driving device 1 has moved to an odd-numbered floor 50a, but the work driving map 51b of an even-numbered floor 50b has been selected. However, since the surrounding conditions of the teaching start position 53 on an odd-numbered floor 50a are similar to those of the teaching start position 53 on an even-numbered floor 50b, the second agreement rate of the processing result of the second matching process that compares the teaching local map with the current local map is approximately 100%. On the other hand, as shown in Figure 10(b), there are differences 62 between the teaching environment image 60 of even-numbered floors 50b and the current environment image 61 of odd-numbered floors 50a. Therefore, the first agreement rate of the processing result of the first matching process, which compares the teaching environment image 60 and the current environment image 61, is calculated to be lower than 100%. The validation evaluation unit 25 calculates, for example, 52% as the overall agreement rate between the processing result of the first matching process and the processing result of the second matching process.

[0119] Next, the evaluation result output unit 26 outputs the validity evaluation result of the reproduction start position by the validity evaluation unit 25 via the display input unit 9 (step S14). For example, in this example, the evaluation result output unit 26 displays the numerical information "52", which is the overall evaluation score, as the overall agreement rate of 52%, on the evaluation result display 47 corresponding to the work travel map 51b of even-numbered floors 50b in the map list 45 displayed in the display area 44 (map selection area) of the display unit 33 of the display input unit 9.

[0120] As described above, when the evaluation score is "52", it exceeds a predetermined agreement rate threshold (for example, 50%), so the validity evaluation unit 25 determines that the validity is met, and the reproduction run control unit 23 does not make it impossible to perform a reproduction run on an odd-numbered floor 50a using the work run map 51b of an even-numbered floor 50b. However, since the evaluation score of "52" is low compared to the perfect score of "100", the operator may have doubts about the validity. Therefore, the evaluation result output unit 26 displays the evaluation result display 47 showing the numerical information "52" in reverse characters to inform the operator that the validity is low. The evaluation result output unit 26 may also output an alarm sound to indicate that the validity is low.

[0121] The operator, by visually inspecting the evaluation result display 47, recognizes the validity of the reproduction start position for the selected work travel map 51b, and if they decide that it is OK to perform the reproduction run using the work travel map 51b, they press the execution key 46 in the display area 44 (Step S15: YES). The reproduction run control unit 23 starts executing the reproduction run mode according to the selected work travel map 51b in response to the operation of the execution key 46 (Step S16).

[0122] Furthermore, if the operator determines that the reproduction run using the work run map 51b is not appropriate and decides to change the work run map 51, they press the back key 48 in the display area 44 (step S15: NO) to re-select the work run map 51 in the map list 45 (step S11).

[0123] As described above, according to the first embodiment, the autonomous driving device 1, which is capable of driving based on manual operation input and autonomous driving based on automatic operation input, comprises a device body 2, a driving unit 3 that drives the device body 2, a measuring unit 6 that measures the distance and angle between the device body 2 and objects around the device body 2, a map creation unit 21 that transforms the coordinates of the distance and angle measured by the measuring unit 6 to create a local map of the area around the device body 2, and, when the teaching driving mode is executed, while the driving unit 3 drives the device body 2 in response to manual operation input, the measuring unit 6 and the map creation unit 21 are activated to connect the local maps, thereby creating a driving path 52 from the teaching start position 53 to the teaching end position 54 and a work driving map 51 around the driving path 52. The system includes a teaching driving control unit 22 and a reproduction driving control unit 23, which are driving control units that store the data and, when the reproduction driving mode is executed, control the driving unit 3 to drive the device body 2 at the reproduction start position by automatic operation input according to the driving route 52 and the work driving map 51; an imaging unit 7 provided on the device body 2 that captures an environmental image showing the surrounding conditions of the device body 2; and a validity evaluation unit 25, which is an evaluation unit that acquires the environmental image captured by the imaging unit 7 when the teaching driving mode is executed as the teaching environmental image 60, acquires the environmental image captured by the imaging unit 7 when the reproduction driving mode is executed as the current environmental image 61, and performs a first matching process by comparing the teaching environmental image 60 and the current environmental image 61 to evaluate the validity of the reproduction start position.

[0124] In other words, the validity evaluation method for the autonomous driving device 1 is a validity evaluation method for evaluating the validity of the reproduction start position in a reproduction run of the autonomous driving device 1, which is capable of driving based on manual operation input and autonomous driving based on automatic operation input, and comprises a measurement step of measuring the distance and angle between the device body 2 of the autonomous driving device 1 and objects around the device body 2, a map creation step of creating a local map of the area around the device body 2 by performing coordinate transformation on the distance and angle measured in the measurement step, and when the driving unit 3 of the autonomous driving device 1 drives the device body 2 in response to manual operation input, the measurement step and the map creation step are performed and the local map is connected so that the driving path 52 from the teaching start position 53 to the teaching end position 54 and the driving The control unit 8's computer is instructed to perform the following steps: a driving control step in which a work driving map 51 is created and stored around the route 52, and when the reproduction driving mode is executed, the driving unit 3 is controlled to drive the device body 2 at the reproduction start position by automatic operation input according to the driving route 52 and the work driving map 51; an imaging step in which an imaging unit 7 provided on the device body 2 captures an environmental image showing the situation around the device body 2; and an evaluation step in which the environmental image captured by the imaging step when the teaching driving mode is executed is acquired as the teaching environmental image 60, the environmental image captured by the imaging step when the reproduction driving mode is executed is acquired as the current environmental image 61, and a first matching process is performed by comparing the teaching environmental image 60 and the current environmental image 61 to evaluate the validity of the reproduction start position.

[0125] With this configuration, in the autonomous driving device 1 according to the first embodiment, even if the operator mistakenly selects a work travel map 51 for a cleaning area 50 other than the cleaning area 50 where the autonomous driving device 1 is located, when the autonomous driving device 1 performs a reproduction run, it is possible to avoid stopping with an error because it becomes impossible to continue the work due to traveling along the wrong work travel map 51, thereby maintaining good accuracy in the reproduction run. Furthermore, in the autonomous driving device 1 according to the first embodiment, when performing a reproduction run, if the operator does not position the autonomous driving device 1 at the correct teaching start position 53, it is possible to avoid stopping with an error because it becomes impossible to continue the work due to traveling along the wrong work travel map 51, thereby maintaining good accuracy in the reproduction run. Accordingly, in the autonomous driving device 1 according to the first embodiment, it is possible to improve the accuracy of the reproduction run without having to pre-install initialization objects along the reproduction run route or accurately position the device body 2 at the teaching start position 53.

[0126] Furthermore, in the autonomous driving device 1 according to the first embodiment, the evaluation result output unit 26, which is an evaluation unit, outputs a first evaluation score as numerical information based on the agreement rate between the teaching environment image 60 and the current environment image 61 obtained by the first matching process.

[0127] This configuration allows for accurate verification of the selection of the work route map 51 using technically reliable image matching processing technology. Furthermore, the operator can easily verify the reproduction start position and the validity of the work route map 51 using numerical information as a result of the matching process.

[0128] Furthermore, in the autonomous driving device 1 according to the first embodiment, the validation evaluation unit 25 acquires a local map created by the map creation unit 21 as the teaching local map at the start of the teaching driving mode, acquires a local map created by the map creation unit 21 as the current local map at the start of the reproduction driving mode, performs a second matching process comparing the teaching local map and the current local map, and evaluates the validity of the reproduction start position by the first matching process and the second matching process.

[0129] This configuration allows for the application of technically reliable image matching processing technology not only to environmental images but also to local maps, enabling the evaluation of the validity of the reproduction start position and the work route map 51. Furthermore, since the same image matching processing technology can be applied to both environmental images and local maps, the overall processing can be simplified and the processing speed can be improved.

[0130] Furthermore, in the autonomous driving device 1 according to the first embodiment, the evaluation result output unit 26 outputs a total evaluation score as numerical information, which is a combination of a first evaluation score based on the agreement rate between the teaching environment image 60 and the current environment image 61 obtained by the first matching process, and a second evaluation score based on the agreement rate between the teaching local map and the current local map obtained by the second matching process.

[0131] This configuration makes it possible to comprehensively verify the validity of the reproduction start position and the work travel map 51 from multiple perspectives, allowing for more accurate validity evaluation results and easier understanding of the evaluation results, thereby reducing the burden on the operator.

[0132] Furthermore, in the autonomous driving device 1 according to the first embodiment, if the reproduction driving control unit 23 evaluates that the validity of the reproduction start position is not met by the validity evaluation unit 25, it will cancel the driving in the reproduction driving mode if it is before the start of driving in the reproduction driving mode.

[0133] This configuration reduces the amount of data stored for environmental images and also makes it easier for the operator to perform operations such as re-selecting the work route map 51 in the event of an error.

[0134] As described above, in the autonomous driving device 1 according to the first embodiment, even if the placement of the reproduction start position of the autonomous driving device 1 for reproduction driving or the selection of the work driving map 51 is incorrect, the placement of the reproduction start position and the selection of the work driving map 51 can be corrected early, thereby suppressing a decrease in work efficiency.

[0135] Next, the autonomous driving device 1 of the second embodiment will be described. In the following description of the second embodiment, components similar to those of the autonomous driving device 1 of the first embodiment will be denoted by the same reference numerals, and detailed descriptions will be omitted.

[0136] In the autonomous driving device 1 of the second embodiment, the validation evaluation unit 25 not only acquires environmental images and local maps at the starting position and performs first matching processing and second matching processing to evaluate the validity of the reproduced starting position based on the processing results, but also acquires multiple environmental images and multiple local maps at predetermined travel distance intervals during travel from the starting position to a predetermined starting section and performs first matching processing and second matching processing to evaluate the validity of the work travel map 51 based on the processing results.

[0137] In the second embodiment, the image acquisition unit 24 acquires an environmental image captured by the imaging unit 7 as a teaching environment image 60 (start position image) when the autonomous driving device 1 is positioned at the teaching start position 53, with the start of the teaching run as the acquisition timing. Then, when the autonomous driving device 1 reaches at least one mark position 55, 56, 57 (see Figures 2 and 3) set at predetermined travel distance (or travel time) intervals during a predetermined start section (or start period) after the start of the teaching run, the image acquisition unit 24 acquires each environmental image captured by the imaging unit 7 as a plurality of teaching environment images 60 (mark position images). Furthermore, when the teaching run control unit 22 ends the teaching run mode and creates the work run map 51, the image acquisition unit 24 stores the plurality of teaching environment images 60, including the start position image and the mark position image, in association with the work run map 51 in the storage unit 16.

[0138] The image acquisition unit 24 may set a predetermined starting section and a predetermined travel distance according to the size of the cleaning area 50, and set the mark positions 55, 56, and 57 based on the said starting section and travel distance. Alternatively, the image acquisition unit 24 may temporarily store environmental images taken between the start of the teaching run and the start section, sequentially determine changes in the environmental images, and set the positions of the environmental images where the rate of change is greater than or equal to a predetermined rate of change threshold as the mark positions 55, 56, and 57.

[0139] Figure 12(a) is a plan view showing an example of a teaching environment image 60 acquired by the image acquisition unit 24 in the autonomous driving device 1. When the autonomous driving device 1 performs teaching navigation on the third floor (odd-numbered floors 50a) as shown in Figure 2, the imaging unit 7 captures a teaching environment image 60 (start position image) as shown in Figure 10(a) when the autonomous driving device 1 is positioned at the teaching start position 53, and captures each teaching environment image 60 (mark position image) when the autonomous driving device 1 reaches each mark position 55, 56, and 57. For example, at mark position 57, which is close to the difference point 62 from the even-numbered floors 50b, the imaging unit 24 captures a teaching environment image 60 (mark position image) as shown in Figure 12(a). The image acquisition unit 24 stores the multiple teaching environment images 60 captured by the imaging unit 7 in the storage unit 16, associating them with the work navigation map 51 of the third floor.

[0140] Furthermore, the image acquisition unit 24 acquires each environmental image captured by the imaging unit 7 as a current environmental image 61 (start position image) when the autonomous driving device 1 is positioned at the start position of the reenactment, with the start of the reenactment as the acquisition timing. Then, when the autonomous driving device 1 reaches at least one mark position 55, 56, 57 (see Figures 2 and 3) set at predetermined distance (or travel time) intervals during a predetermined start section (or start period) after the start of the reenactment, the imaging unit 24 acquires multiple environmental images 61 (mark position images) captured by the imaging unit 7 as a plurality of current environmental images 61 (mark position images).

[0141] Figure 12(b) is a plan view showing an example of a current environment image 61 acquired by the image acquisition unit 24 in the autonomous driving device 1. When the autonomous driving device 1 performs a recreation drive of the 4th floor (even-numbered floor 50b) as shown in Figure 3, the imaging unit 7 captures a current environment image 61 (starting position image) as shown in Figure 10(b) when the autonomous driving device 1 is positioned at the recreation starting position, and captures each current environment image 61 (mark position image) when the autonomous driving device 1 reaches each mark position 55, 56, and 57. For example, at mark position 57, which is close to the difference point 62 from the odd-numbered floor 50a, the imaging unit 7 captures a current environment image 61 (mark position image) as shown in Figure 12(b).

[0142] In the second embodiment, the map creation unit 21 acquires a local map created when the autonomous driving device 1 is positioned at the teaching start position 53 at the start of the teaching run as the teaching time local map. Then, after the start of the teaching run, it acquires a plurality of local maps created when the autonomous driving device 1 reaches at least one mark position 55, 56, 57 (see Figures 2 and 3) set at predetermined driving distance (or driving time) intervals during a predetermined starting section (or starting period), as multiple teaching time local maps. Furthermore, when the teaching run control unit 22 ends the teaching run mode and creates the work run map 51, the map creation unit 21 stores the plurality of teaching time local maps of the teaching start position 53 and each mark position 55, 56, 57 in association with the work run map 51 in the storage unit 16.

[0143] Furthermore, the map creation unit 21 acquires a local map created when the autonomous driving device 1 is positioned at the start of the reproduction run as the current local map. Then, after the start of the reproduction run, it acquires multiple local maps created when the autonomous driving device 1 reaches at least one mark position 55, 56, 57 (see Figures 2 and 3), which is set at predetermined distance (or time) intervals, during a predetermined start section (or start period).

[0144] The validation evaluation unit 25 first performs a first matching process on the teaching environment image 60 of the teaching start position 53 and the current environment image 61 of the reproduction start position at the start of the reproduction run, and a second matching process on the teaching local map of the teaching start position 53 and the current local map of the reproduction start position, and calculates an overall processing result (e.g., overall agreement rate) to evaluate the validity of the reproduction start position. Furthermore, during the execution of the reproduction run, when each mark position 55, 56, and 57 is reached, the validation evaluation unit 25 performs a first matching process on the teaching environment image 60 and the current environment image 61 of each mark position 55, 56, and 57, and a second matching process on the teaching local map and the current local map of each mark position 55, 56, and 57, and calculates an overall processing result (e.g., overall agreement rate) to evaluate the validity of the work run map 51 selected for the reproduction run.

[0145] For example, the autonomous driving device 1 has been moved to an odd-numbered floor 50a, such as the third floor, but the work driving map 51b of an even-numbered floor 50b, such as the fourth floor, may be selected for the reproduction driving. In this case, since the surrounding conditions of the teaching start position 53 on the odd-numbered floor 50a and the surrounding conditions of the teaching start position 53 on the even-numbered floor 50b are similar, the validity evaluation unit 25 calculates a second matching rate of almost 100% by performing a second matching process that compares the teaching time local map of the teaching start position 53 with the current local map of the reproduction start position.

[0146] On the other hand, even if there are differences 62 between the teaching environment image 60 at the teaching start position 53 on an even-numbered floor 50b and the current environment image 61 at the reproduction start position on an odd-numbered floor 50a, as shown in Figure 10(b), if the size of the differences 62 is relatively small compared to the entire teaching environment image 60 and current environment image 61, the validity evaluation unit 25 calculates a first agreement rate that is not too low compared to 100% by performing a first matching process that compares the teaching environment image 60 and the current environment image 61, for example, calculating a first agreement rate of about 70%. In this case, the validity evaluation unit 25 evaluates that the validity of the reproduction start position is satisfied if it calculates an overall agreement rate of a predetermined agreement rate threshold or higher as a result of the overall processing of the first matching process and the second matching process.

[0147] Next, as the autonomous driving device 1 proceeds with the reenactment drive and reaches each of the mark positions 55, 56, and 57, the validation evaluation unit 25 performs a first matching process by comparing the teaching environment image 60 and the current environment image 61 of each mark position 55, 56, and 57, and also performs a second matching process by comparing the teaching local map and the current local map of each mark position 55, 56, and 57. At this time, when the autonomous driving device 1 approaches the difference point 62 between odd-numbered floor 50a and even-numbered floor 50b and reaches a mark position 57, the validation evaluation unit 25 performs a first matching process by comparing the teaching environment image 60 (see Figure 12(a)) and the current environment image 61 (see Figure 12(b)) of the mark position 57, and also performs a second matching process by comparing the teaching local map and the current local map of the mark position 57.

[0148] Here, since the mark position 57 is in front of the difference point 62, and the surrounding conditions of the mark position 57 on odd-numbered floors 50a are the same as those surrounding the mark position 57 on even-numbered floors 50b, the validity evaluation unit 25 calculates a second agreement rate of almost 100% by performing a second matching process that compares the local map of the mark position 57 at the time of teaching with the current local map of the mark position 57. On the other hand, as shown in Figures 12(a) and 12(b), the size of the difference 62 is relatively large compared to the overall teaching environment image 60 and current environment image 61 when comparing the teaching environment image 60 and the current environment image 61 with the teaching environment image 60 and the current environment image 61. The validity evaluation unit 25 calculates a first agreement rate lower than 100% through a first matching process that compares the teaching environment image 60 and the current environment image 61, for example, by calculating a first agreement rate below a predetermined agreement rate threshold.

[0149] The validation evaluation unit 25 then evaluates that the validity of the work route map 51 is satisfied if, as a result of the overall processing of the first matching process and the second matching process for each mark position 55, 56, and 57, it calculates an overall matching rate that is equal to or greater than a predetermined matching rate threshold, while evaluating that the validity of the work route map 51 is not satisfied if it calculates an overall matching rate that is less than the predetermined matching rate threshold.

[0150] In the second embodiment, even if the validity evaluation unit 25 evaluates that the overall match rate based on the first match rate obtained by the first matching process and the second match rate obtained by the second matching process at the reproduction start position is less than a predetermined match rate threshold, the reproduction driving control unit 23 may start the reproduction driving without making it impossible (canceling) to perform the reproduction driving. Subsequently, if the reproduction driving control unit 23 evaluates that the validity of the work driving map 51 is satisfied based on the overall processing results of the first matching process and the second matching process for each mark position 55, 56, and 57, it will continue the reproduction driving as is. On the other hand, if it evaluates that the validity of the work driving map 51 is not satisfied, it will make it impossible to perform the reproduction driving of the work driving map 51. For example, after starting driving in reproduction driving mode, the reproduction driving control unit 23 controls the autonomous driving device 1 to drive back to the reproduction start position and to stop at the reproduction start position.

[0151] Furthermore, in the second embodiment, as timing for acquiring the teaching environment image 60 and the teaching local map during teaching driving, in addition to the teaching start position 53 and the mark positions 55, 56, and 57 set at predetermined driving distance intervals during the start section, sub-mark positions may also be set at the positions where the learning key 40, which acts as a recording switch, is operated. That is, according to the sub-mark position operated by the learning key 40, the image acquisition unit 24 acquires the teaching environment image 60 and the map creation unit 21 acquires the teaching local map, and stores them in the storage unit 16 in association with the work driving map 51. Note that the image acquisition unit 24 may acquire the teaching environment image 60 and the map creation unit 21 may acquire the teaching local map not only at the sub-mark positions but also at positions before and after them.

[0152] Furthermore, during the reenactment run, when the autonomous driving device 1 reaches the sub-mark position used during teaching, the image acquisition unit 24 acquires the current environment image 61, and the map creation unit 21 acquires the current local map. The validity evaluation unit 25 then performs a first matching process of the teaching environment image 60 and the current environment image 61 at the sub-mark position, and a second matching process of the current local map and the current environment image 61, calculates an overall processing result (e.g., overall match rate), and evaluates the validity of the work run map 51 selected for the reenactment run.

[0153] For example, when the autonomous driving device 1 reaches a location within the cleaning area 50 where the surrounding conditions (layout, etc.) differ during a teaching run, the operator can use the learning key 40 to acquire a teaching environment image 60 and a teaching local map, using that location as a sub-mark location. This allows the validation evaluation unit 25 to evaluate the validity of the work run map 51 through a first matching process using the teaching environment image 60 at locations with different surrounding conditions and a second matching process using the teaching local map. As a result, the validation evaluation unit 25 can more appropriately compare the differences in surrounding conditions within the cleaning area 50.

[0154] Furthermore, the validation unit 25 may automatically identify locations where the surrounding conditions differ by comparing the work travel maps 51 of multiple cleaning areas 50 with each other, and set sub-mark positions. In this case, the validation unit 25 may compare not only the shape of the work travel map 51, but also points of change in the travel route 52.

[0155] In this case, in order to acquire the environmental image 60 for teaching the sub-mark position, the image acquisition unit 24 temporarily stores environmental images of each point during the teaching run, and when the validity evaluation unit 25 sets the sub-mark position, it stores the environmental image corresponding to that sub-mark position as the teaching environmental image 60. Alternatively, during the reproduction run, when the autonomous driving device 1 reaches the sub-mark position set during the teaching run, the image acquisition unit 24 acquires an environmental image and stores it as the teaching environmental image 60.

[0156] The autonomous driving device 1 may register each work travel map 51 of the multiple cleaning areas 50 to an external server (not shown), and the external server may compare each work travel map 51 with each other to set the sub-mark positions.

[0157] This allows the autonomous driving device 1 to accumulate data (environmental images) from successful reenactments of each cleaning area 50 as it performs the reenactment multiple times. Therefore, even if an error occurs in selecting the work route map 51, the autonomous driving device 1 can respond appropriately.

[0158] As described above, according to the second embodiment, the autonomous driving device 1 acquires a plurality of teaching environment images 60 by capturing environmental images with the imaging unit 7 at predetermined travel distance intervals during a predetermined start period after starting to drive in teaching driving mode, and acquires each local map created by the map creation unit 21 as a plurality of teaching local maps. The validity evaluation unit 25 evaluates that the validity of the reproduction start position is satisfied if, when executing the reproduction driving mode, the overall match rate based on the first match rate by the first matching process and the second match rate by the second matching process at the reproduction start position is less than a predetermined match rate threshold, and acquires a plurality of current environment images 61 by capturing environmental images with the imaging unit 7 at predetermined travel distance intervals during a predetermined start section after starting to drive in reproduction driving mode, and acquires each local map created by the map creation unit 21 as a plurality of current local maps, and the overall match rate based on the first match rate by the first matching process of the plurality of current environment images 61 and the second match rate by the second matching process of the plurality of current local maps at a predetermined start section is equal to or greater than a predetermined match rate threshold.

[0159] With this configuration, even when the autonomous driving device 1 repeatedly zigzags within the same cleaning area 50, it focuses on matching environmental images and local maps during the initial start period. This reduces the amount of data to be stored compared to storing environmental images and local maps over the entire travel path 52. Furthermore, even if the selection of the cleaning area 50 is correct, but the reproduction start position and direction (angle) are significantly off, the device can still start the reproduction run and evaluate the validity of the reproduction start position and the work travel map 51. Moreover, if the validity of the work travel map 51 is evaluated after the reproduction run has started, the reproduction run can be continued. Therefore, the autonomous driving device 1 can easily start the reproduction run without having to precisely position its reproduction start position at the teaching start position 53.

[0160] Furthermore, according to the first and second embodiments, when the validity evaluation unit 25 evaluates that the validity of the reproduction start position is not met, the reproduction driving control unit 23 controls the driving unit 3 to drive the device body 2 back to the reproduction start position and stop at the reproduction start position, if driving in reproduction driving mode has already started.

[0161] This configuration allows the autonomous driving device 1 to reduce the amount of environmental image data it stores, and also makes it easier for the operator to perform operations such as re-selecting the work driving map 51 when an error occurs.

[0162] Next, the autonomous driving device 1 of the third embodiment will be described. In the following description of the third embodiment, components similar to those of the autonomous driving device 1 of the first and second embodiments will be denoted by the same reference numerals, and detailed descriptions will be omitted.

[0163] In the autonomous driving device 1 of the first embodiment described above, the validation evaluation unit 25 uses the start of the reproduced driving as the evaluation timing, and evaluates the work driving map 51 selected from the map list 45 in the display area 44 (map selection area) of the display unit 33 of the display input unit 9. It performs a first matching process between the teaching environment image 60 and the current environment image 61, and a second matching process between the teaching local map and the current local map, and evaluates the validity of the reproduction start position for the selected work driving map 51 based on the processing results. The evaluation result output unit 26 then displays the overall evaluation score, which is the validity evaluation result for the selected work driving map 51, on the evaluation result display 47 corresponding to the work driving map 51 to be evaluated in the map list 45 of the display area 44.

[0164] In contrast, in the autonomous driving device 1 of the third embodiment, the validity evaluation unit 25 evaluates each of the multiple work driving maps 51 already stored in the memory unit 16, that is, each work driving map 51 displayed in the map list 45 of the display area 44 (map selection area) of the display input unit 9, and performs a first matching process between the teaching environment image 60 and the current environment image 61, and a second matching process between the teaching local map and the current local map, and evaluates the validity of the reproduction start position for each work driving map 51 based on the processing results of each work driving map 51. The evaluation result output unit 26 then outputs a total evaluation score for each of the multiple work driving maps 51, which is obtained by combining the first evaluation score from the first matching process and the second evaluation score from the second matching process, on the evaluation result display 47 corresponding to each work driving map 51 in the map list 45 of the display area 44.

[0165] For example, the autonomous driving device 1 has been moved to an odd-numbered floor 50a, such as the 3rd floor, but the work driving map 51b of an even-numbered floor 50b, such as the 4th floor, may be selected for the purpose of replicating the driving experience. In this case, as shown in Figure 8, the evaluation result output unit 26 displays numerical information of "52" as the overall evaluation score in the map list 45 in the display area 44, which is the map selection area, on the evaluation result display 47 corresponding to the work driving map 51a of the odd-numbered floor 50a, and numerical information of "94" as the overall evaluation score in the evaluation result display 47 corresponding to the work driving map 51b of the even-numbered floor 50b.

[0166] The validation unit 25 may use the time when a work driving map 51 is selected from the map list 45 in the display area 44 as the evaluation timing, as in the first embodiment. Alternatively, it may use the time when the replay driving mode starts, for example, when the main switch 30 is switched to "automatic" and the system switches to the replay driving mode, and the work driving maps 51 already stored in the storage unit 16 are displayed in a selectable list in the map list 45 in the display area 44 as the evaluation timing.

[0167] Furthermore, when the validity evaluation unit 25 evaluates the validity of the reproduction start position for multiple work travel maps 51 and the evaluation result output unit 26 outputs the overall evaluation score to the evaluation result display 47, the reproduction travel control unit 23 automatically selects the work travel map 51 with the highest overall evaluation score and controls the system to perform a reproduction run according to that work travel map 51. For example, the reproduction travel control unit 23 automatically selects the work travel map 51 with the highest overall evaluation score when it has finished evaluating the validity of the reproduction start position for multiple work travel maps 51, or in response to the operation of the automatic selection key 49 in the display area 44.

[0168] Furthermore, if the work travel map 51 selected from the map list 45 satisfies the validity of the start position for reproduction, the reproduction travel control unit 23 may apply the work travel map 51 as is to the reproduction travel. On the other hand, if the selected work travel map 51 does not satisfy the validity of the start position for reproduction, the unit may automatically select the work travel map 51 with the highest overall evaluation score, regardless of the work travel map 51 selection operation.

[0169] Furthermore, in the third embodiment, if the validation evaluation unit 25 evaluates that the selected work travel map 51 does not meet the validity requirements during execution at the start of the replayed run, it performs a validity evaluation simulation on other work travel maps 51 already stored in the storage unit 16. Based on this simulation, the validation evaluation unit 25 performs a first matching process between the teaching environment image 60 and the current environment image 61, and a second matching process between the teaching local map and the current local map for the other work travel maps 51, and evaluates the validity of each work travel map 51 based on the processing results for each work travel map 51.

[0170] Furthermore, if the validation evaluation unit 25 performs the above-described simulation, the reproduction driving control unit 23 automatically replaces the other work driving maps 51 with the work driving map 51 that exceeds the score threshold and has the highest overall evaluation score, and controls the system to perform a reproduction drive according to that work driving map 51. At this time, the reproduction driving control unit 23 may pre-select, according to the operator's operation, whether to return the autonomous driving device 1 to the reproduction start position and restart the reproduction drive, or to continue the reproduction drive from the position of the autonomous driving device 1 at the time the work driving map 51 was replaced, or it may ask the operator via the display input unit 9 or the like during the simulation to make a selection.

[0171] In the above-described embodiment, the image acquisition unit 24 acquires environmental images captured by the forward camera 7a of the imaging unit 7 as the teaching environment image 60 and the current environment image 61 when the teaching driving mode or the reproduction driving mode is executed, and the validity evaluation unit 25 performs a first matching process on the teaching environment image 60 and the current environment image 61 captured by the forward camera 7a. However, the present invention is not limited to this example.

[0172] In other examples, the image acquisition unit 24 acquires environmental images captured not only by the front camera 7a of the imaging unit 7 but also by the rear camera 7b as the teaching environment image 60 and the current environment image 61 when the teaching driving mode or the reproduction driving mode is executed. The validity evaluation unit 25 performs a first matching process not only on the teaching environment image 60 and the current environment image 61 captured by the front camera 7a but also on the teaching environment image 60 and the current environment image 61 captured by the rear camera 7b, and evaluates the validity of the reproduction start position based on the processing results of both first matching processes.

[0173] Figure 13 is a plan view showing an example of an environmental image captured by the rear camera 7b of the imaging unit 7 in the autonomous driving device 1. The rear camera 7b is equipped with, for example, a wider-angle lens than the front camera 7a, and captures a distorted environmental image like the one shown in Figure 13.

[0174] Figure 14 is a plan view showing an example of how image processing is performed on an environmental image captured by the rear camera 7b of the imaging unit 7 in the autonomous driving device 1. The validation evaluation unit 25 performs coordinate transformation and distortion correction processing on the environmental image captured by the rear camera 7b to obtain an image like that shown in Figure 14(a), and also performs semantic segmentation processing to obtain an image color-coded by attribute as shown in Figure 14(b), and further extracts a portion of the central image as shown in Figure 14(c). When the operator is operating the handle 4a of the control unit 4, the operator will be reflected in the environmental image of the rear camera 7b during teaching driving, as shown by the dashed line in Figure 13. However, by replacing a portion of the environmental image with an environmental image acquired during reproduction driving, or by combining them, an environmental image with the operator removed can be obtained as shown in Figure 14(a). By applying a wide-angle lens to the rear camera 7b, it is possible to capture an environmental image with a sufficient imaging range other than the operator behind it.

[0175] The validation evaluation unit 25 performs the same image processing on the teaching environment image 60 and the current environment image 61 captured by the front camera 7a, and on the teaching environment image 60 and the current environment image 61 captured by the rear camera 7b. This allows for faster processing and reduced usage of the storage unit 16, even when multiple image processing operations are performed simultaneously, thereby enabling accurate validation evaluation.

[0176] Furthermore, even when performing the first matching process on the environmental image captured by the front camera 7a and the environmental image captured by the rear camera 7b, the first matching process may be performed not only on the environmental image at the starting position, but also on each mark position 55, 56, and 57 of the starting period, in the same manner as in the third embodiment described above. In this case, by using the environmental image captured by the rear camera 7b, the first matching process for the starting period can be completed earlier.

[0177] Furthermore, to simplify image processing, the timing of environmental image acquisition by the image acquisition unit 24, for example, at the start of the teaching run, may be notified by a display on the display unit 33 or by an audio output from the speaker 32 to move the operator out of the field of view of the front camera 7a and the rear camera 7b. At this time, the image acquisition unit 24 may first detect, using sensors, that there are no people within the field of view of the front camera 7a and the rear camera 7b, and then capture the teaching environment image of the teaching start position 53 using the front camera 7a and the rear camera 7b. Alternatively, the teaching run control unit 22 may enable the start of the teaching run after the capture of the teaching environment image is complete.

[0178] Furthermore, the reproduction driving control unit 23 may, when performing a reproduction drive, store the current environment image 61 captured by the rear camera 7b at the reproduction start position and each mark position 55, 56, 57 in the storage unit 16 as a teaching environment image 60 for subsequent reproduction drives.

[0179] In the embodiment described above, the cleaning unit 5 is described as a dry cleaning mechanism having a cleaning member 13 such as a roll brush, a suction unit 14, and a bucket, but the present invention is not limited to this example. In other examples, the cleaning unit 5 may be a wet cleaning mechanism having a cleaning member, a cleaning liquid supply unit, a suction unit, and a squeegee.

[0180] In other embodiments, the validity evaluation unit 25 may calculate the agreement rate and evaluation score of the weighted evaluation results by adding weights to the notation information such as characters and logos in the environmental images during the first matching process of the teaching environment image 60 and the current environment image 61.

[0181] In this case, before performing semantic segmentation processing on the teaching environment image 60 and the current environment image 61, the validation evaluation unit 25 detects and extracts notation information images containing notation information such as characters and logos from the teaching environment image 60 and the current environment image 61. Then, in addition to the first matching process of the teaching environment image 60 and the current environment image 61, the validation evaluation unit 25 performs a first matching process on each notation information image of the teaching environment image 60 and the current environment image 61. Through the first matching process of the notation information images, the validation evaluation unit 25 determines the differences in notation information such as characters and logos between the teaching environment image 60 and the current environment image 61, and whether or not such notation information exists. The validation evaluation unit 25 can obtain a naturally weighted evaluation result by applying the evaluation result (matching rate) of the first matching process of the notation information images to the evaluation result (matching rate) of the first matching process of the teaching environment image 60 and the current environment image 61.

[0182] In other embodiments, the validation evaluation unit 25 may, at the start of the reproduced run or during the run within the start period, determine that the validity of the reproduced start position or the work run map 51 is not met, and then make an error determination and display the teaching environment image 60 and the current environment image 61 on the display unit 9 display 33 in a comparable manner.

[0183] Alternatively, the autonomous driving device 1 may be equipped with a communication unit 19 (see Figure 6) that can communicate with an external server (not shown) via a network such as the Internet, and the validation evaluation unit 25 may, at the start of the reproduced driving or during the execution of the start period, evaluate that the validity of the reproduced starting position or the work driving map 51 is not met, send the teaching environment image 60 and the current environment image 61 to the external server along with an error notification.

[0184] The external server is configured to remotely monitor the reproduced driving and performs comparison processing of the received teaching environment image 60 and the current environment image 61, as well as analysis processing of the content and reason for errors. The external server may also display the results of the comparison and analysis processing on the screen to notify of errors. Furthermore, the external server is configured to communicate with the mobile terminal (not shown) of the operator or administrator of the autonomous driving device 1 via a network such as the internet, and may also send the results of the comparison and analysis processing to the mobile terminal via email or an application to notify of errors.

[0185] In this way, if the autonomous driving device 1 detects an error in the placement of the reproduction start position or the selection of the work driving map 51, the autonomous driving device 1, an external server, or a mobile terminal will notify the operator or administrator of the error, allowing them to take action to correct the mistake.

[0186] Furthermore, when issuing an error notification, the display input unit 9 or the like may prompt the operator to select whether to return the autonomous driving device 1 to the reproduction start position and restart the reproduction run, or to replace it with the work run map 51 that obtained the highest match rate and continue the reproduction run from that position. Alternatively, when issuing an error notification, the autonomous driving device 1 may automatically select the option to replace it with the work run map 51 that obtained the highest match rate and continue the reproduction run from that position.

[0187] Furthermore, the autonomous driving device 1 may have pre-set priorities for various response settings when an error is detected. This allows the autonomous driving device 1 to automatically select and apply the highest-priority response setting when it is unable to determine the appropriate response setting, such as when the operator or administrator is absent or when communication with an external server or mobile terminal is impossible.

[0188] Thus, according to the autonomous driving device 1 of the present invention, the evaluation result of the validity of the reproduction start position relative to the work driving map 51 can be output and made known to the operator without increasing costs, thereby simplifying the operator's work and enabling smooth progress of tasks such as cleaning associated with the reproduction driving.

[0189] Furthermore, the present invention may be modified as appropriate, provided that it does not contradict the gist or idea of ​​the invention as can be inferred from the claims and the specification as a whole, and autonomous driving devices and validation methods involving such modifications are also included in the technical concept of the present invention. [Industrial applicability]

[0190] The present invention can be suitably used in autonomous driving devices capable of both manual and automatic operation for performing cleaning work over relatively large areas such as hotel floors and office floors. [Explanation of Symbols]

[0191] 1. Autonomous driving device 2. Main unit of the device 3. Running section 5 Cleaning Department 6 Measurement Unit 7 Imaging Unit 7a Front camera 7b Rear camera 8 Control Unit 9 Display Input Section 16 Memory section 20 Mode switching section 21 Mapmaking Department 22 Instructional Driving Control Unit 23 Reproduction Driving Control Unit 24 Image acquisition unit 25 Validation Department 26 Evaluation Result Output Unit 33 Display 40 Learning Keys 44 Display Area 46 Execution Key 47. Posting of evaluation results 49 Automatic Selection Key 50 Cleaning Areas 50a Odd-numbered floors 50b Even-numbered floors 51, 51a, 51b Work route map 52. Route 53. Instruction start position 54 End of Instruction Position Mark positions 55, 56, 57 60. Instructional environment image 61 Current Environment Image

Claims

1. An autonomous driving device capable of driving based on manual operation input and autonomous driving based on automatic operation input, The main body of the device, A traveling unit that moves the main body of the device, A measuring unit that measures the distance and angle between the device body and objects surrounding the device body, A map creation unit creates a local map of the area around the main body of the device by performing coordinate transformation on the distance and angle measured by the measurement unit, During the execution of the teaching driving mode, while the driving unit drives the main body of the device in response to manual operation input, the measurement unit and the map creation unit are activated to connect the local maps, thereby creating and storing a driving path from the teaching start position to the teaching end position and a work driving map of the area around the driving path. When the reproduction driving mode is executed, the driving control unit controls the driving unit to drive the main unit of the device at the reproduction start position according to the driving route and the work driving map, The device body includes an imaging unit that captures an environmental image showing the surrounding conditions of the device body, An evaluation unit that acquires the environmental image captured by the imaging unit during the execution of the teaching driving mode as the teaching environmental image, acquires the environmental image captured by the imaging unit during the execution of the reproduction driving mode as the current environmental image, performs a first matching process comparing the teaching environmental image and the current environmental image to evaluate the validity of the reproduction start position, Equipped with, The autonomous driving device is characterized in that the evaluation unit acquires the local map created by the map creation unit as the teaching local map at the start of the teaching driving mode, acquires the local map created by the map creation unit as the current local map at the start of the reproduction driving mode, performs a second matching process comparing the teaching local map and the current local map, and evaluates the validity of the reproduction start position by the first matching process and the second matching process.

2. The autonomous driving device according to claim 1, characterized in that the evaluation unit outputs a first evaluation score as numerical information based on the agreement rate between the teaching environment image and the current environment image obtained by the first matching process.

3. The autonomous driving device according to claim 1, characterized in that the evaluation unit outputs a total evaluation score as numerical information, which is obtained by combining a first evaluation score based on the agreement rate between the teaching environment image and the current environment image obtained by the first matching process, and a second evaluation score based on the agreement rate between the teaching local map and the current local map obtained by the second matching process.

4. From the start of driving in the teaching driving mode, during a predetermined starting section, the imaging unit captures the environmental image at predetermined driving distance intervals to acquire multiple teaching environmental images, and the map creation unit acquires each of the local maps created as multiple teaching local maps. The autonomous driving device according to claim 1, characterized in that, when the reproduction driving mode is executed, if the overall matching rate based on the first matching rate by the first matching process and the second matching rate by the second matching process at the reproduction start position is less than a predetermined matching rate threshold, the evaluation unit acquires a plurality of current environmental images by capturing the environmental images with the imaging unit at predetermined driving distance intervals between the start of driving in the reproduction driving mode and the predetermined start section, and acquires each of the local maps created by the map creation unit as a plurality of current local maps, and evaluates that the validity of the reproduction start position is satisfied when the overall matching rate based on the first matching rate by the first matching process of the plurality of current environmental images at the predetermined start section and the second matching rate by the second matching process of the plurality of current local maps is equal to or greater than a predetermined matching rate threshold.

5. The autonomous driving device according to claim 1 or 2, characterized in that when the driving control unit evaluates that the validity of the reproduction start position is not met by the evaluation unit, if it is before driving in the reproduction driving mode, it stops driving in the reproduction driving mode, and if it has already started driving in the reproduction driving mode, it controls the driving unit to drive the device body back to the reproduction start position and stop at the reproduction start position.

6. A validity evaluation method for evaluating the validity of the start position of a reproduced run in a reproduced run of an autonomous driving device capable of driving based on manual operation input and autonomous driving based on automatic operation input, A measurement step of measuring the distance and angle between the device body of the autonomous driving device and objects surrounding the device body, A mapping step is performed to create a local map of the area around the device body by performing a coordinate transformation on the distance and angle measured in the measurement step, During the execution of the teaching driving mode, while the driving unit of the autonomous driving device drives the device body in response to manual operation input, the measurement step and the map creation step are executed to connect the local maps, thereby creating and storing a driving path from the teaching start position to the teaching end position and a work driving map of the area around the driving path. A driving control step in which, when the reproduction driving mode is executed, the driving unit is controlled to drive the main body of the device at the reproduction start position by automatic operation input according to the driving route and the work driving map, The imaging step involves capturing an environmental image showing the surrounding conditions of the device body using an imaging unit provided on the device body, An evaluation step which involves acquiring the environmental image captured by the imaging step during the execution of the teaching driving mode as the teaching environmental image, acquiring the environmental image captured by the imaging step during the execution of the reproduction driving mode as the current environmental image, and performing a first matching process by comparing the teaching environmental image and the current environmental image to evaluate the validity of the reproduction start position, Have the computer run it, The validation method is characterized in that the evaluation step involves acquiring the local map created by the map creation step as the teaching local map at the start of the teaching driving mode, acquiring the local map created by the map creation step as the current local map at the start of the reproduction driving mode, performing a second matching process that compares the teaching local map and the current local map, and evaluating the validity of the reproduction start position by the first matching process and the second matching process.

Images