Cleaning system, unmanned flying device and autonomous cleaning device

Abstract

To provide a cleaning system capable of easily and reliably collecting and cleaning widely scattered collection objects.SOLUTION: A cleaning system 1 comprises an autonomous traveling cleaning device 4 and an unmanned flying device 2. The autonomous traveling cleaning device 4 comprises a device main body 30, a traveling unit 31, a cleaning unit 32 which performs cleaning on a traveling route, a control unit 38 which functions as an environment map acquisition unit 60 acquiring an environment map of a cleaning area, and a communication unit 40 which performs radio communication with the unmanned flying device 2. The unmanned flying device 2 comprises an imaging unit 12, and a communication unit 18 which performs radio communication with the autonomous traveling cleaning device 4. The imaging unit 12 images the cleaning area from above to generate imaging data when the unmanned flying device 2 flies above the cleaning area. The communication unit 18 transmits the imaging data to the autonomous traveling cleaning device 4. The control unit 38 functions as a data reflection unit 61 which reflects the imaging data on the environment map by associating the imaging data imaged by the imaging unit 12 with the environment map.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a cleaning system equipped with an autonomously traveling cleaning device capable of performing autonomous traveling and cleaning automatically, an unmanned flying device capable of cooperating with this autonomously traveling cleaning device, and this autonomously traveling cleaning device. [Background technology]

[0002] Conventionally, some autonomous cleaning devices perform autonomous cleaning by autonomously traveling in a cleaning area and automatically collecting and cleaning up litter such as dust and fallen leaves on the floor or ground of the cleaning area. Some autonomous cleaning devices perform autonomous cleaning in cooperation with an unmanned flying device.

[0003] For example, in the cleaning system of Patent Document 1, a self-propelled robotic cleaning device that cleans the floor surface and an aircraft device that flies through the air perform cleaning operations in cooperation with each other. The aircraft device performs an operation of dropping a cleaning target that is located higher than the floor surface onto the floor surface at a cooperation location within the cleaning area using a cleaning aid, and transmits a notification regarding the operation of dropping the cleaning target onto the floor surface to the robotic cleaning device via a communication unit. After a predetermined time has elapsed since receiving the notification regarding the operation of dropping the cleaning target onto the floor surface at the cooperation location within the cleaning area, the robotic cleaning device performs the cooperation operation by traveling on the floor surface of the cooperation location and cleaning. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-91326 A Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional technology such as the cleaning system of Patent Document 1, an aircraft device uses a cleaning aid to drop cleaning objects such as garbage that are higher than the floor surface at a coordinated location within a cleaning area onto the floor surface. However, cleaning objects that are higher than the floor surface do not necessarily fall into the coordinated location, so cleaning objects that do not fall into the coordinated location end up remaining on the floor surface without being cleaned.

[0006] Also, for example, in parks or parking lots where trees are planted side by side, many fallen leaves may fly around and scatter over a wide area depending on the season. In such a wide-ranging cleaning area, it is very difficult to clean and collect fallen leaves and other items to be collected, so efficient automatic cleaning is required. Furthermore, in parking lots, vehicles are parked randomly, and fallen leaves and other items to be collected may fall on the ground of the vehicles. blind spot Since the items to be collected are often scattered and hidden in the parking lot, it is very time-consuming to visually search for the items to be collected in the parking lot. Therefore, it is very difficult to automatically collect and clean the items to be collected that are scattered over a wide area in the parking lot.

[0007] The present invention has been developed in consideration of the problems described above, and an object of the present invention is to provide a cleaning system, an unmanned flying device, and an autonomously moving cleaning device that can easily, reliably, and automatically collect and clean items scattered over a wide area. [Means for solving the problem]

[0008] In order to solve the above problems, the first cleaning system of the present invention is a cleaning system including an autonomously traveling cleaning device capable of performing automatic traveling cleaning by traveling autonomously and cleaning automatically, and further including an unmanned flying device, the autonomously traveling cleaning device including a device main body, a traveling unit that travels the device main body, a cleaning unit that cleans on the traveling path of the device main body, an environmental map acquisition unit that acquires an environmental map of the cleaning area where the automatic traveling cleaning is performed, and a first communication unit that wirelessly communicates with the unmanned flying device, the unmanned flying device including an imaging unit and a second communication unit that wirelessly communicates with the autonomously traveling cleaning device, the imaging unit images the cleaning area from above when the unmanned flying device flies above the cleaning area to generate imaging data, and the second communication unit wirelessly communicates with the first communication unit to transmit the imaging data to the autonomously traveling cleaning device, The autonomous cleaning device and / or the unmanned flying device includes an object discrimination unit that discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit, The autonomous cleaning device includes: The collection target object and the non-collection target object discriminated by the target object discriminating unit The imaging data is reflected in the environmental map by associating the imaging data with the environmental map.

[0009] According to the first cleaning system of the present invention, by reflecting the image data of the cleaning area taken by the unmanned flying device on the environmental map, it is possible to obtain information on blind spots that the autonomous cleaning device cannot see, and to obtain an environmental map showing the cleaning area in more detail. Therefore, the autonomous cleaning device can automatically collect and clean the collection objects scattered over a wide area in the cleaning area easily and reliably.

[0010] In order to solve the above problems, in the second cleaning system of the present invention, the autonomously traveling cleaning device is characterized in that it includes a cleaning plan creation unit that creates a cleaning plan for performing the autonomously traveling cleaning to clean the collection objects based on the environmental map, and an autonomously traveling cleaning control unit that performs the autonomously traveling cleaning based on the cleaning plan.

[0011] According to the second cleaning system of the present invention, it is possible to obtain information about blind spots that the autonomous cleaning device cannot see, and to create a cleaning plan that allows for more reliable collection so as not to leave any items to be collected in the cleaning area.

[0012] In order to solve the above problems, in the third cleaning system of the present invention, the unmanned flying device is equipped with a flight control unit that controls a collection operation that utilizes downwash during flight to blow and gather the collection objects at a predetermined collection position within the cleaning area, the imaging unit, after performing the collection operation, images the cleaning area including the collection position to generate the imaging data, the data reflection unit reflects the collection position imaged in the imaging data on the environmental map, and the cleaning plan creation unit creates the cleaning plan for performing the automatic driving cleaning so as to prioritize cleaning of the collection objects at the collection position based on the environmental map.

[0013] According to the third cleaning system of the present invention, the autonomous cleaning device can clean most of the objects to be collected in the cleaning area by cleaning the collection location where the objects to be collected were collected, thereby improving cleaning efficiency. Therefore, the objects to be collected in the cleaning area can be collected more easily and reliably so that none of them are left behind.

[0014] In order to solve the above problems, in a fourth cleaning system of the present invention, the autonomously traveling cleaning device is equipped with an obstacle recognition unit that recognizes obstacles based on measurement results of the positional relationship of obstacles around the device main body, and the object discrimination unit is equipped with a collection position setting unit that discriminates the non-collection objects on the floor or ground of the cleaning area based on recognition information of the obstacle recognized by the obstacle recognition unit, and sets the collection position on one side of the non-collection objects discriminated by the object discrimination unit based on the image data or the recognition information.

[0015] According to the fourth cleaning system of the present invention, by utilizing non-collected objects at the collection location, the collected objects can be more reliably blown together, and can be more easily and more reliably collected and cleaned so that no collected objects are left behind in the cleaning area.

[0016] In order to solve the above problem, the first unmanned flying device of the present invention comprises an imaging unit that images the cleaning area from above when flying above the cleaning area to generate imaging data, an object discrimination unit that discriminates between collection objects and non-collection objects on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit, a collection position setting unit that sets a collection position on one side of the non-collection objects discriminated by the object discrimination unit based on the imaging data, a flight control unit that controls a collection operation that uses downwash during flight to blow the collection objects to the collection position within the cleaning area, and a communication unit that wirelessly communicates with an autonomously traveling cleaning device, wherein the imaging unit images the cleaning area including the collection position after performing the collection operation to generate the imaging data, and the communication unit wirelessly communicates with the autonomously traveling cleaning device to transmit the imaging data captured by the imaging unit after performing the collection operation to the autonomously traveling cleaning device.

[0017] According to the first unmanned flying device of the present invention, the collection location where the collection objects are collected can be provided to the autonomously traveling cleaning device, so that the autonomously traveling cleaning device can clean most of the collection objects in the cleaning area by cleaning the collection location, thereby improving the cleaning efficiency. Therefore, the collection objects in the cleaning area can be collected more easily and reliably so that none are left behind.

[0018] In order to solve the above problem, the second unmanned flying device of the present invention further includes a cleaning device detection unit that detects the position of the autonomously moving cleaning device based on the imaging data, and the collection position setting unit sets the collection position on one side of the non-collection objects identified by the object discrimination unit that are closest to the autonomously moving cleaning device.

[0019] According to the second unmanned flying device of the present invention, by using non-collection objects close to the autonomous cleaning device as collection locations, the collection objects can be more reliably blown together, making it easier and more reliable to collect the objects so that they are not left behind in the cleaning area.

[0020] In order to solve the above problems, in the third unmanned flying device of the present invention, the collection position setting unit is erected on the floor or ground of the cleaning area, recognizes landmarks that separate the cleaning area based on the imaging data, and sets the collection position on one side of the recognized landmark.

[0021] According to the third unmanned flying device of the present invention, even if there are no non-collection objects in the cleaning area to collect the collection objects, landmarks for collecting the collection objects can be easily installed in the cleaning area, making it possible to collect the collection objects.

[0022] In order to solve the above problem, in the fourth unmanned flying device of the present invention, the cleaning device detection unit is characterized in that it detects the position of the autonomously traveling cleaning device by detecting a beacon signal transmitted by the autonomously traveling cleaning device.

[0023] In addition, in the fifth unmanned flying device of the present invention, the cleaning device detection unit detects the position of the autonomously moving cleaning device by detecting infrared rays emitted by a laser range finder to measure the positional relationship of surrounding obstacles as the beacon signal.

[0024] According to the fourth or fifth unmanned flying device of the present invention, the position of the autonomously traveling cleaning device can be grasped more easily and reliably, and non-collection objects close to the autonomously traveling cleaning device can be used as collection locations for the collection objects, so that the collection objects can be more reliably gathered together.

[0025] In order to solve the above problems, the sixth unmanned flying device of the present invention is characterized in that it is equipped with a moving object discrimination unit that discriminates a moving object moving on the floor or ground of the cleaning area, and the flight control unit aborts flight when the moving object discrimination unit discriminates the moving object.

[0026] According to the sixth unmanned flying device of the present invention, when the unmanned flying device is flying to take aerial images of a cleaning area or to collect collection items, if a moving object such as a person is present, the flight will be interrupted, thereby improving safety.

[0027] In order to solve the above problems, the first autonomous traveling cleaning device of the present invention is an autonomous traveling cleaning device capable of performing autonomous traveling and automatic cleaning, comprising: a device main body; a traveling unit that travels the device main body; a cleaning unit that cleans along the traveling path of the device main body; an environmental map acquisition unit that acquires an environmental map of the cleaning area in which the autonomous traveling and cleaning is performed; and a communication unit that receives imaging data from the unmanned flying device by wirelessly communicating with the unmanned flying device, the imaging data generated by the unmanned flying device flying above the cleaning area and imaging the cleaning area from above; An object discrimination unit that discriminates between objects to be collected and objects to be non-collected on the floor or ground of the cleaning area based on the imaging data received from the unmanned flying device by the communication unit; and and a data reflection unit that reflects the imaging data in the environmental map by associating the imaging data with the environmental map.

[0028] According to the first autonomous cleaning device of the present invention, by reflecting the image data of the cleaning area taken by the unmanned flying device on the environmental map, it is possible to obtain information on blind spots that the autonomous cleaning device cannot see, and to obtain an environmental map showing the cleaning area in more detail. Therefore, the autonomous cleaning device can automatically collect and clean the collection objects scattered over a wide area in the cleaning area easily and reliably. Effect of the Invention

[0029] According to the present invention, objects to be collected that are scattered over a wide area can be automatically collected and cleaned easily and reliably. [Brief description of the drawings]

[0030] [Figure 1] 1 is a schematic diagram showing a configuration of a cleaning system according to an embodiment of the present invention. [Diagram 2] 1 is a block diagram showing a schematic configuration of a cleaning system according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a block diagram showing the electrical configuration of the cleaning system according to the embodiment of the present invention. [Figure 4] A plan view showing an example of a cleaning area that is subject to manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. [Diagram 5] A plan view showing an example of a case in which collection items are scattered in a cleaning area that is the subject of manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. [Figure 6] A plan view showing an example of a case in which an unmanned flying device manually flies over a cleaning area littered with collection items in a cleaning system according to an embodiment of the present invention. [Figure 7] A plan view showing an example of a cleaning system according to an embodiment of the present invention, in which an unmanned flying device performs manual flight to collect collection items at a collection position. [Figure 8] 13 is a flowchart illustrating an example of the operation of manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. [Figure 9] A flowchart showing an example of the automatic traveling and cleaning operation of an autonomous traveling cleaning device after manual flight of an unmanned flying device in a cleaning system according to an embodiment of the present invention. [Figure 10] A flowchart showing an example of the automatic flight operation of an unmanned flying device and the automatic driving and cleaning operation of an autonomous driving cleaning device in a cleaning system according to an embodiment of the present invention. [Figure 11] A flowchart showing another example of the automatic flight operation of an unmanned flying device and the automatic driving and cleaning operation of an autonomous driving cleaning device in a cleaning system according to an embodiment of the present invention. [Figure 12]A plan view showing an example of the measurement range of the measurement unit of an autonomously moving cleaning device relative to a cleaning area that is the target of automatic flight of the unmanned flying device in a cleaning system according to an embodiment of the present invention. [Figure 13] A plan view showing an example of the imaging range of an unmanned flying device relative to a cleaning area that is the target of automatic flight of the unmanned flying device in a cleaning system according to an embodiment of the present invention. [Figure 14] A plan view showing an example of an unmanned flying device automatically flying over a cleaning area littered with collection items in a cleaning system according to an embodiment of the present invention. [Figure 15] A plan view showing an example of a cleaning system according to an embodiment of the present invention, in which an unmanned flying device performs automatic flying to collect collection items at a collection position. [Figure 16] 1 is a plan view showing an example of a cleaning area where no non-collection objects are to be set in a cleaning system according to an embodiment of the present invention. FIG. [Figure 17] 11 is a plan view showing an example in which landmarks for setting collection positions are provided in a cleaning area in which there are no non-collection targets and in which collection positions are to be set in a cleaning system according to an embodiment of the present invention. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] Hereinafter, embodiments of the present invention will be described with reference to the drawings. The following embodiments are preferred specific examples of the present invention and disclose various preferred techniques, but the technical scope of the present invention is not limited to these aspects.

[0032] A cleaning system 1 according to an embodiment of the present invention will be described. As shown in Fig. 1, the cleaning system 1 includes an unmanned flying device 2, a control device 3 for controlling the unmanned flying device 2, and an autonomously traveling cleaning device 4 capable of performing autonomous traveling and cleaning. The unmanned flying device 2 and the autonomously traveling cleaning device 4 are configured to be able to cooperate with each other.

[0033] The unmanned flying device 2 of this embodiment will be described.

[0034] The unmanned flying device 2 is configured as a so-called drone, for example, which performs flying operations by being controlled by a control device 3 or according to a preset program. As shown in Fig. 1, the unmanned flying device 2 includes a device main body 10 for housing each part, a flying unit 11 for flying the device main body 10, and an imaging unit 12 for capturing images of the surroundings.

[0035] As shown in Figure 2, the unmanned flying device 2 also includes a positioning unit 13 that acquires position information of the unmanned flying device 2, a flight detection unit 14 that detects the flight status of the unmanned flying device 2, and a power supply unit 15 that supplies power to each part of the unmanned flying device 2 and controls monitoring of the remaining charge and charging of the battery (not shown).

[0036] Furthermore, the unmanned flying device 2 includes a control unit 16 that provides overall control of each unit and function of the unmanned flying device 2 (such as flight by the flying unit 11 and imaging by the imaging unit 12), and a memory unit 17 that stores a flight plan, which is the flight program of the flying unit 11, imaging data captured by the imaging unit 12, etc. The unmanned flying device 2 includes a communication unit 18 (second communication unit) for wireless communication with external devices such as the pilot device 3 and the autonomously traveling cleaning device 4.

[0037] Next, each part of the unmanned flying device 2 will be described. The flying unit 11 has a plurality of (for example, four) flying wings 11a on the upper part of the device body 10, and also has a plurality of flight motors 11b that respectively rotate and drive the plurality of flying wings 11a. The flying unit 11 controls the flight altitude, flight direction, flight attitude, flight speed, etc. of the unmanned flying device 2 by controlling the rotation speed of each flight motor 11b using the control unit 16.

[0038] The imaging unit 12 is provided at the bottom of the device body 10, and captures an aerial image of the area below the unmanned flying device 2 to generate imaging data. The imaging unit 12 is composed of, for example, a CCD (Charge Coupled Device) camera. The imaging unit 12 generates and outputs, as imaging data, the image of the captured aerial image itself and / or image analysis data of the captured aerial image.

[0039] The positioning unit 13 uses a satellite positioning system such as the Global Navigation Satellite System (GNSS) or the Global Positioning System (GPS) to acquire position information of the unmanned flying device 2. The flight detection unit 14 is composed of a gyro sensor and the like, and detects flight conditions such as changes in attitude and acceleration of the unmanned flying device 2 during flight.

[0040] The power supply unit 15 includes a battery (power source) and a charging circuit mounted inside the device main body 10, and when connected to an external power source the battery is charged and power is supplied to each part of the unmanned flying device 2.

[0041] The control unit 16 is composed of a computer device such as a CPU (Central Processing Unit), and is connected to a storage unit 17 including a ROM (Read Only Memory), a RAM (Random Access Memory), an SSD (Solid State Drive), a flash memory, etc. The control unit 16 is also connected to each part of the unmanned flying device 2, such as the flying unit 11, the imaging unit 12, the positioning unit 13, the flight detection unit 14, and the power supply unit 15.

[0042] The control unit 16 is also connected to be able to communicate with external devices via the communication unit 18. The communication unit 18 performs wireless communication with, for example, the pilot device 3 and the autonomously traveling cleaning device 4 using a communication standard such as Bluetooth (registered trademark). The communication unit 18 receives flight instructions from the pilot device 3, receives instructions from the pilot device 3 to the imaging unit 12 to capture aerial images, and transmits imaging data of the aerial images captured by the imaging unit 12 to the pilot device 3 and the autonomously traveling cleaning device 4.

[0043] The memory unit 17 stores programs and data for controlling each part and various functions of the unmanned flying device 2, and the control unit 16 executes arithmetic processing based on the programs and data stored in the memory unit 17, thereby controlling each part and various functions of the unmanned flying device 2. For example, the control unit 16 executes the programs stored in the memory unit 17, thereby operating as a flight control unit 50, a flight plan creation unit 51, an imaging control unit 52, an object discrimination unit 53, a collection position setting unit 54, a cleaning device detection unit 55, and a moving object discrimination unit 56, as shown in FIG.

[0044] When the communication unit 18 receives a flight instruction from the pilot device 3, the flight control unit 50 controls the multiple flight motors 11b of the flight unit 11 in accordance with the flight instruction to rotate and drive each of the flight wings 11a, and controls the manual flight of the flight unit 11 to control the flight altitude, flight direction, flight attitude, flight speed, etc. of the unmanned flying device 2. When manually flying the flight unit 11, the flight control unit 50 can collect collection objects on the floor or ground of the cleaning area toward a specified collection direction (collection position) by using downwash generated by the rotation of the flight wings 11a of the flight unit 11.

[0045] The flight control unit 50 also controls the flight unit 11 to perform automatic flight according to a flight plan, which is a flight program pre-stored in the memory unit 17. The flight plan includes information such as the flight path of the unmanned flying device 2, and the flight altitude, flight direction, flight attitude, and flight speed at each position on the flight path.

[0046] For example, the flight plan includes a pre-flight plan created in advance to confirm the cleaning area to be cleaned by the autonomously traveling cleaning device 4. The pre-flight plan is created to perform a pre-flight in which the unmanned flying device 2 hovers at a predetermined position in the cleaning area to a flight altitude that overlooks the cleaning area in order to capture an aerial image of the cleaning area by the imaging unit 12 of the unmanned flying device 2. In the pre-flight plan, the imaging position and imaging altitude on the flight path of the unmanned flying device 2 are set as the imaging timing of the imaging unit 12. The pre-flight plan may be created to perform a single hover to capture an aerial image at one imaging position in the cleaning area, or may be created to perform a moving flight to each imaging position and hovering at each imaging position in order to capture an aerial image at multiple imaging positions in the cleaning area.

[0047] When the communication unit 18 receives an automatic flight instruction from the pilot device 3, the flight control unit 50 controls the flight unit 11 based on the preliminary flight plan to perform a preliminary flight of the unmanned flying device 2.

[0048] The flight plan creation unit 51 creates a collection flight plan so as to perform an automatic collection flight that assists in the collection of objects on the floor or ground of a cleaning area by downwash generated by the rotation of the flying wings 11a of the flying unit 11. Since the topography and size of a cleaning area differ from one cleaning area to another, and the scattered state of objects to be collected on a cleaning area differs from cleaning to cleaning, the flight plan creation unit 51 creates a collection flight plan for each cleaning.

[0049] The flight plan creation unit 51 creates a collection flight plan based on the collection objects and non-collection objects identified by the object discrimination unit 53 and the collection positions set by the collection position setting unit 54. Specifically, the flight plan creation unit 51 creates a collection flight plan based on a flight path that flies so as to blow and gather the collection objects to the collection positions by downwash, while avoiding the non-collection objects.

[0050] Furthermore, the flight plan creation unit 51 creates a collection flight plan so that, at the end of the automatic collection flight, a post-flight is performed in which the unmanned flying device 2 hovers at a predetermined position in the cleaning area to a flight altitude that overlooks the cleaning area in order to capture aerial images of the cleaning area using the imaging unit 12 of the unmanned flying device 2 and generate imaging data for analyzing the cleaning area. The post-flight may operate in the same manner as the pre-flight.

[0051] When the flight control unit 50 performs a preliminary flight of the unmanned flying device 2 to generate imaging data of the cleaning area, the flight plan creation unit 51 creates a collection flight plan based on this imaging data. In addition, when the flight plan creation unit 51 creates a collection flight plan, the flight control unit 50 controls the flying unit 11 based on this collection flight plan to perform an automatic collection flight of the unmanned flying device 2.

[0052] The imaging control unit 52 controls the imaging unit 12 to capture aerial images below the unmanned flying device 2 so that the pilot device 3 can check the flight status while the unmanned flying device 2 is flying manually or automatically. At this time, the imaging control unit 52 causes the imaging unit 12 to periodically capture still images or videos to generate imaging data for confirmation, and causes the communication unit 18 to transmit the imaging data to the pilot device 3.

[0053] When the imaging control unit 52 receives an imaging instruction from the pilot device 3, the imaging control unit 52 controls the imaging unit 12 in response to the imaging instruction to capture an aerial image below the unmanned flying device 2 in order to generate imaging data for analyzing the cleaning area. In addition, when the unmanned flying device 2 automatically flies according to the preliminary flight plan, the imaging control unit 52 controls the imaging unit 12 in response to the imaging position and imaging altitude set in the preliminary flight plan to capture an aerial image below the unmanned flying device 2 in order to generate imaging data for analyzing the cleaning area. At this time, the imaging control unit 52 causes the imaging unit 12 to capture a still image to generate imaging data for analysis, and causes the communication unit 18 to transmit the imaging data to the pilot device 3 and the autonomously traveling cleaning device 4.

[0054] In order to create a collection flight plan by the flight plan creation unit 51, the object discrimination unit 53 discriminates collection objects and non-collection objects on the floor or ground of the cleaning area based on the image data for analysis captured by the imaging unit 12. For example, the object discrimination unit 53 uses an image analysis technique represented by a semantic segmentation method based on the difference in shape and color between the floor or ground, collection objects such as fallen leaves, and non-collection objects such as walls to discriminate and discriminate collection objects and non-collection objects from the floor or ground, and further discriminate and discriminate collection objects and non-collection objects. The object discrimination unit 53 calculates the position information of the discriminated collection objects and non-collection objects in the cleaning area.

[0055] The collection position setting unit 54 determines and sets a collection position where the collection objects should be collected in the cleaning area, based on the results of the discrimination of the collection objects and non-collection objects discriminated by the object discrimination unit 53. For example, the collection position setting unit 54 sets a collection position on one side of the non-collection objects that are closer to the autonomous traveling cleaning device 4, among the discriminated non-collection objects, closer to the autonomous traveling cleaning device 4. At this time, the collection position setting unit 54 determines the non-collection objects for which a collection position should be set, based on the position of the autonomous traveling cleaning device 4 detected by the cleaning device detection unit 55.

[0056] Alternatively, the collection position setting unit 54 may determine a concentration position where the objects to be collected are most concentrated, and set a collection position on one side of the objects to be collected that is closest to the concentration position. The collection position setting unit 54 may set one collection position for the cleaning area, or may set multiple collection positions.

[0057] The cleaning device detection unit 55 detects the position of the autonomously traveling cleaning device 4 relative to the unmanned flying device 2. For example, the unmanned flying device 2 is equipped with a beacon receiver and receives beacon signals such as infrared or Bluetooth (registered trademark) transmitted by the autonomously traveling cleaning device 4, and the cleaning device detection unit 55 detects the beacon signal transmitted by the autonomously traveling cleaning device 4.

[0058] The cleaning device detection unit 55 detects the beacon signal to recognize the position of the autonomously traveling cleaning device 4 relative to the unmanned flying device 2, and recognizes the position of the autonomously traveling cleaning device 4 in the cleaning area. For example, the cleaning device detection unit 55 detects infrared rays transmitted by an LRF 34a (described later) of the measurement unit 34 of the autonomously traveling cleaning device 4 as a beacon signal to recognize the position of the autonomously traveling cleaning device 4. The cleaning device detection unit 55 may reflect the detected position of the autonomously traveling cleaning device 4 in the imaging data captured by the imaging unit 12.

[0059] The moving object discrimination unit 56 discriminates moving objects such as people that move or exist around (within a predetermined distance from) the unmanned flying device 2. For example, the moving object discrimination unit 56 may be composed of a human sensor or the like to discriminate moving objects, or may discriminate moving objects using image analysis technology based on imaging data captured by the imaging unit 12. For example, when the moving object discrimination unit 56 operates while the unmanned flying device 2 is performing automatic flight and discriminates a moving object, the flight control unit 50 temporarily suspends the automatic flight and performs hovering on the spot. After that, when the moving object discrimination unit 56 no longer discriminates a moving object, the flight control unit 50 resumes automatic flight.

[0060] The control device 3 of this embodiment will be described.

[0061] The control device 3 controls the unmanned flying device 2 via wireless communication, and is configured, for example, as shown in Fig. 2, with a control unit 20 and a display unit 21. The control device 3 also has a control unit 22 that controls each unit and each function of the control device 3, and a storage unit 23 that stores various information. The control device 3 also has a communication unit 24 for wireless communication with external devices such as the unmanned flying device 2.

[0062] The control unit 22 is composed of a computer device such as a CPU, and is connected to a storage unit 23 including a ROM, a RAM, an SSD, a flash memory, etc. The control unit 22 is also connected to each part of the control device 3, such as the control unit 20 and the display unit 21.

[0063] The control unit 22 is also connected to be able to communicate with external devices via the communication unit 24. The communication unit 24 performs wireless communication with the unmanned flying device 2, for example, using a communication standard such as Bluetooth (registered trademark). The communication unit 24 transmits, for example, flight instructions corresponding to the operation of the control unit 20 and instructions to capture aerial images of the cleaning area to the unmanned flying device 2, and receives imaging data of the aerial images captured by the imaging unit 12 from the unmanned flying device 2.

[0064] The memory unit 23 stores programs and data for controlling each part and various functions of the control device 3, and the control unit 22 performs overall control of each part and various functions of the control device 3 by executing calculations based on the programs and data stored in the memory unit 23.

[0065] The control unit 20 is composed of, for example, left and right joysticks, and outputs flight instructions for the unmanned flying device 2 in response to the operation of the joysticks, and transmits them to the unmanned flying device 2 via the communication unit 24. The flight instructions indicate, for example, the flight altitude, flight direction, flight attitude, flight speed, etc. of the unmanned flying device 2.

[0066] The display unit 21 is composed of a touch panel or the like, and displays information related to the operation of the unmanned flying device 2. For example, the display unit 21 receives imaging data of an aerial image of the cleaning area captured by the imaging unit 12 of the unmanned flying device 2 from the unmanned flying device 2 via the communication unit 24 and displays it. This allows the worker to control the flight of the unmanned flying device 2 using the control unit 20 while checking the aerial image of the cleaning area displayed on the display unit 21.

[0067] In addition, in order to obtain imaging data for analysis of the cleaning area during manual flight of the unmanned flying device 2, the display unit 21 displays an imaging button that can be operated to instruct the imaging unit 12 to capture an image, and causes the communication unit 24 to transmit an imaging instruction to the unmanned flying device 2 in response to operation of the imaging button.

[0068] Furthermore, in order to perform automatic flight (pre-flight or automatic collection flight) of the unmanned flying device 2, the display unit 21 operably displays an automatic flight button that instructs automatic flight in the cleaning area, and transmits an automatic flight instruction to the unmanned flying device 2 via the communication unit 24 in response to operation of the automatic flight button. The display unit 21 may also be provided with buttons that can individually instruct the unmanned flying device 2 to perform a pre-flight or automatic collection flight.

[0069] The control device 3 may be replaced by a tablet terminal or a smartphone, in which case the control unit 20 and the image capture button may be configured as a GUI (Graphical User Interface) on a touch panel. Furthermore, the control device 3 may be configured to be operable using a voice input operation technology.

[0070] An autonomous traveling cleaning device 4 according to an embodiment of the present invention will now be described.

[0071] The autonomous traveling cleaning device 4 travels through a wide area such as a park or parking lot as a cleaning area, collecting and cleaning up garbage such as dust and fallen leaves scattered on the floor or ground of the cleaning area as collection targets. As an example, as shown in Fig. 4, a parking lot as a cleaning area has multiple parking spaces, in which vehicles, which are not collection targets, are parked, and the parking lot has walls, which are not collection targets. In such a parking lot, the autonomous traveling cleaning device 4 cleans the cleaning area where fallen leaves, which are collection targets, are scattered, as shown in Fig. 5.

[0072] As shown in Figure 1, the autonomously moving cleaning device 4 comprises a device main body 30 for housing each component, a running unit 31 for moving the device main body 30, a cleaning unit 32 for cleaning the floor or ground below the device main body 30, and a running operation unit 33 for accepting manual operation of the running unit 31.

[0073] The autonomous traveling cleaning device 4 also includes a measurement unit 34 that measures the positional relationship between the device body 30 and surrounding walls and non-collection targets such as vehicles and obstacles (e.g., ornaments), and an imaging unit 35 that captures images of the surroundings of the device body 30. As shown in Fig. 2, the autonomous traveling cleaning device 4 also includes an operation display unit 36 ​​for operating various functions of the autonomous traveling cleaning device 4 and displaying various information, and a power supply unit 37 that supplies power to each unit of the autonomous traveling cleaning device 4 and controls the monitoring of the remaining capacity of the battery (not shown) and charging.

[0074] Furthermore, the autonomous traveling cleaning device 4 includes a control unit 38 that controls each unit and each function of the autonomous traveling cleaning device 4 (traveling by the traveling unit 31, cleaning work by the cleaning unit 32, measurement by the measurement unit 34, etc.), and a memory unit 39 that stores a cleaning plan consisting of traveling data of the traveling unit 31 and cleaning data of the cleaning unit 32. The autonomous traveling cleaning device 4 also includes a communication unit 40 (first communication unit) for wireless communication with external devices such as the unmanned flying device 2.

[0075] Next, an overview of the operation of the autonomous traveling cleaning device 4 will be described.

[0076] Before performing automatic traveling cleaning of the cleaning area, the autonomous traveling cleaning device 4 acquires an environmental map of the cleaning area in advance, and creates a cleaning plan including traveling data and cleaning data of the automatic traveling cleaning based on the environmental map. The autonomous traveling cleaning device 4 associates the environmental map and the cleaning plan with the cleaning area and stores them in the storage unit 39. The traveling data indicates the traveling path and traveling state of the automatic traveling cleaning, and the cleaning data indicates the cleaning state of the automatic traveling cleaning. The autonomous traveling cleaning device 4 sets a predetermined collection position in the cleaning area, and creates a cleaning plan so that when a collection object is collected at this collection position, the collection object at this collection position is preferentially cleaned. The autonomous traveling cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 by the control unit 38 to perform automatic traveling cleaning based on the environmental map and cleaning plan associated with the cleaning area to be cleaned.

[0077] The cleaning plan may be composed of, for example, a number of steps along a travel route, with each step being associated with travel data and cleaning data. The interval between steps may be set to a predetermined time interval (e.g., 25 ms) or may be set to a predetermined travel distance (e.g., 0.5 m). The cleaning plan may be stored in association with the time elapsed since the start of cleaning for each step.

[0078] The travel data includes, for example, self-position data on the travel route of the travel unit 31 (X and Y coordinates indicating the self-position on the environmental map, and the angle relative to the direction of the starting position), a steering flag (straight ahead, left turn, right turn), travel speed [m / s] in the travel direction, and turning speed [deg / s], and the travel route can be illustrated based on the travel data. Alternatively, the travel speed may be switched to one of a plurality of speed stages, and the travel data may be stored by converting the stepwise speed of the travel speed into a number (for example, 8 stages from 0 to 7).

[0079] The cleaning data includes, for example, the rotation speed of brush 32a of cleaning unit 32 and the suction volume of suction unit 32b. The rotation speed of brush 32a and the suction volume of suction unit 32b may be switched to one of a plurality of strength levels, and the cleaning data may be stored by converting the strength levels of the rotation speed and suction volume into numbers (e.g., three levels from 0 to 2, or five levels from 0 to 4, etc.). The cleaning data may include the operation / stop and rotation speed of brush 32a and suction unit 32b.

[0080] Next, each component of the autonomous traveling cleaning device 4 will be described.

[0081] The running unit 31 is provided at the bottom of the device body 30, and includes, for example, a pair of front wheels 31a as drive wheels and rear wheels 31b as auxiliary wheels. The front wheels 31a are provided on both sides (left and right sides) in the device width direction at the front side in the traveling direction, and include a pair of left and right running drive motors (not shown) and a front wheel rotation encoder (not shown). The running unit 31 drives the running drive motor to rotate the front wheels 31a to move the device body 30 forward, and stops the device body 30 by stopping the rotation of the front wheels 31a. The running speed of the autonomous traveling cleaning device 4 (running unit 31) is adjusted (accelerated or decelerated) by controlling the drive of the running drive motor. Note that the running unit 31 may move the device body 30 backward by the running drive motor rotating the front wheels 31a in the reverse direction.

[0082] The pair of front wheels 31a are equipped with a steering rotation encoder (not shown), and the device body 30 is pivoted to the left or right by rotating the pair of front wheels 31a in opposite directions. For example, the device body 30 is turned to the left by rotating the right front wheel 31a forward and the left front wheel 31a in the reverse direction, while the device body 30 is turned to the right by rotating the right front wheel 31a in the reverse direction and the left front wheel 31a in the forward direction. The turning radius of the autonomous cleaning device 4 (traveling unit 31) is adjusted by controlling the rotation of each of the pair of front wheels 31a.

[0083] The rear wheel 31b is rotatably provided at the center in the width direction of the device on the rear side in the traveling direction. The rear wheel 31b rotates in response to the movement of the device body 30 driven by the pair of front wheels 31a. Note that, in the present embodiment, an example is described in which the front wheel 31a serving as a driving wheel and the rear wheel 31b serving as an auxiliary wheel are provided, but the present invention is not limited to this example. For example, in other embodiments, the front wheel 31a serving as an auxiliary wheel and the rear wheel 31b serving as a driving wheel may be provided.

[0084] When the autonomous cleaning device 4 performs autonomous traveling and cleaning of a cleaning area, the traveling unit 31 operates under the control of the control unit 38 based on the traveling data of the environmental map and cleaning plan associated with the cleaning area.

[0085] The cleaning unit 32 is provided below the device body 30 and is configured to clean the floor or ground below the device body 30. When the autonomous cleaning device 4 performs autonomous traveling and cleaning of a cleaning area, the cleaning unit 32 operates under the control of the control unit 38 based on the environmental map associated with the cleaning area and the cleaning data of the cleaning plan.

[0086] The cleaning unit 32 is composed of a dry cleaning mechanism that collects and cleans garbage such as dust and fallen leaves on the floor or ground as collection objects. The cleaning unit 32 includes a brush 32a that sweeps the floor or ground below the device body 30 to collect collection objects on the floor or ground, a suction unit 32b that sucks up the collection objects collected by the brush 32a, and a collection unit 32c that collects and accumulates the collection objects sucked up by the suction unit 32b. The cleaning unit 32 also includes a brush motor (not shown) that rotates the brush 32a on the floor or ground and a suction motor (not shown) that operates suction by the suction unit 32b.

[0087] The brushes 32a are arranged side by side on both sides (left and right sides) in the width direction of the device, and the left and right brushes 32a are configured to be large enough to contact each other at the center side in the width direction of the device and protrude outward in the width direction of the device. The left brush 32a rotates clockwise when viewed from above, and the right brush 32a rotates counterclockwise when viewed from above, rotating from the front to the rear at the center side in the width direction. This causes the objects to be collected in front of the brush 32a to be collected toward the center side in the width direction and then toward the rear. The suction unit 32b is provided behind the brush 32a and is configured, for example, by a suction blower. The suction unit 32b is connected to the collection unit 32c and discharges the objects to be collected that it has sucked into the collection unit 32c.

[0088] The traveling operation unit 33 is provided on the upper rear side of the device body 30, and includes a handle and a throttle that can be manually operated by an operator. Note that the traveling operation unit 33 accepts manual operations by an operator when, for example, creating an environmental map, but does not accept manual operations other than the operation of an emergency stop button when performing automatic traveling and cleaning.

[0089] The travel operation unit 33 converts the steering amount of the handle by the operator into an electric signal, and outputs the electric signal to the travel drive motor to drive it, thereby rotating the pair of front wheels 31a and making the device main body 30 turn left (left turn) or right (right turn). The turning radius of the autonomous traveling cleaning device 4 (travel unit 31) is adjusted according to the steering amount of the handle of the travel operation unit 33.

[0090] The travel operation unit 33 also converts the amount of throttle rotation (opening) by the operator into an electric signal, and outputs the electric signal to the travel drive motor to drive it, thereby rotating the pair of front wheels 31a and moving the device body 30 forward in the travel direction. The travel speed of the autonomous traveling cleaning device 4 (travel unit 31) is adjusted according to the amount of throttle rotation of the travel operation unit 33.

[0091] The measurement unit 34 includes a laser range finder (LRF) 34a that measures position information (e.g., angle and distance with respect to the traveling direction of the device body 30) between the device body 30 and non-collection targets such as surrounding walls and obstacles. The measurement unit 34 measures position information between the device body 30 and non-collection targets, for example, at predetermined timings (e.g., every 25 ms) while the device body 30 is traveling. The LRF 34a is disposed in a cutout portion formed by cutting out the front side of the device body 30 in the left-right direction, and has a measurement range on the front side and both the left and right sides as shown in FIG. 12. In FIG. 12, the measurement range of the LRF 34a is shown as a sector around the LRF 34a.

[0092] The measurement unit 34 also includes an obstacle detection unit (not shown) such as an ultrasonic sensor or an infrared sensor that detects the presence or absence of non-collectible objects such as walls or obstacles within a predetermined distance from the device main body 30. The obstacle detection unit may be in operation at all times while the device main body 30 is traveling, and detects non-collectible objects that may come into contact with the device main body 30 as the device continues to travel, and outputs an alarm signal.

[0093] The imaging unit 35 is provided on the upper part of the device body 30, and captures an image of the surroundings of the autonomous traveling cleaning device 4 to generate surrounding image data. The imaging unit 35 is composed of, for example, a CCD camera.

[0094] The operation display unit 36 ​​is provided on the upper rear side of the device body 30, and includes, for example, a key switch, an emergency stop button, and a touch panel. The operation display unit 36 ​​also includes an environmental map creation button, an environmental map input button, and an automatic traveling and cleaning execution button. The operation display unit 36 ​​may be configured as an operation display panel attached to the device body 30, or may be configured as a tablet terminal or the like detachably attached to the device body 30. The operation display unit 36 ​​configured as a tablet terminal or the like is connected to the control unit 38 via wireless communication by the communication unit 40, and the autonomous traveling and cleaning device 4 can be remotely controlled.

[0095] The key switch is configured to be switchable between on and off. By switching the key switch on, power is supplied from the power supply unit 37 to each unit and the autonomous traveling cleaning device 4 operates, whereas by switching the key switch off, power supply from the power supply unit 37 to each unit is stopped and the operation of the autonomous traveling cleaning device 4 is stopped. By operating the emergency stop button, the operation of each unit of the autonomous traveling cleaning device 4, in particular the automatic traveling and cleaning, is forcibly stopped (braked).

[0096] The touch panel displays various screens in response to control signals from the control unit 38, and transmits operation signals based on touch operations on each screen to the control unit 38. For example, when the key switch is turned on to operate the autonomous traveling cleaning device 4, the touch panel displays an operation screen that allows the autonomous traveling cleaning of the autonomous traveling cleaning device 4 to be operated. The operation screen identifiably displays, for example, an environmental map of the cleaning area where the autonomous traveling cleaning is performed, a traveling route, traveling data of the cleaning plan, and cleaning data. In addition, the operation screen may be configured to allow the traveling speed of the traveling unit 31, the number of rotations of the brush 32a of the cleaning unit 32, the suction amount of the suction unit 32b, and the like to be operated.

[0097] The environmental map creation button, when operated, causes the control unit 38 to create an environmental map of the cleaning area based on the measurement results of the measurement unit 34 while the autonomously traveling cleaning device 4 is traveling in the cleaning area. At this time, the autonomously traveling cleaning device 4 may manually travel in the cleaning area in response to the operation of the travel operation unit 33, or may simply automatically travel in the cleaning area based on the measurement results of the measurement unit 34. The environmental map input button, when operated, causes the control unit 38 to input environmental map data from an external device. The automatic traveling and cleaning execution button, when operated, causes the control unit 38 to execute automatic traveling and cleaning of the autonomously traveling cleaning device 4. The environmental map creation button, the environmental map input button, and the automatic traveling and cleaning execution button may be displayed operably on the operation screen of the touch panel.

[0098] The power supply unit 37 includes a battery (power source) and a charging circuit mounted inside the device body 30, and charges the battery by connecting to an external power source, and also supplies power to each component of the autonomous traveling cleaning device 4. The power supply unit 37 may output a signal indicating the remaining charge of the battery to the control unit 38.

[0099] The control unit 38 is composed of a computer such as a CPU, and is connected to a storage unit 39 including a ROM, a RAM, an SSD, a flash memory, etc., as shown in Fig. 2. The control unit 38 is also connected to each unit of the autonomous traveling cleaning device 4, such as the traveling unit 31, the cleaning unit 32, the traveling operation unit 33, the measurement unit 34, the operation display unit 36, and the power supply unit 37.

[0100] The control unit 38 is also connected to be able to communicate with external devices via the communication unit 40. The communication unit 40 performs wireless communication with external devices such as the operation display unit 36, which is configured by a tablet terminal or the like separate from the device main body 30, and the operator's control device 3, using a communication standard such as Bluetooth (registered trademark).

[0101] The storage unit 39 stores programs and data for controlling each part and various functions of the autonomous traveling cleaning device 4, and the control unit 38 executes arithmetic processing based on the programs and data stored in the storage unit 39 to centrally control each part and various functions. For example, the control unit 38 executes a program stored in the storage unit 39 to operate as an environmental map acquisition unit 60, a data reflection unit 61, an object discrimination unit 62, an obstacle recognition unit 63, a collection position setting unit 64, a cleaning plan creation unit 65, an automatic traveling cleaning control unit 66, and a moving object discrimination unit 67, as shown in FIG. 3. This allows the autonomous traveling cleaning device 4 to autonomously travel and work automatically according to a cleaning plan stored in advance. The storage unit 39 stores an environmental map and a cleaning plan in association with a cleaning area to be subjected to automatic traveling cleaning, and may store an environmental map and a cleaning plan for each of a plurality of cleaning areas.

[0102] The environmental map acquisition unit 60 acquires an environmental map of the cleaning area where the autonomous traveling and cleaning is performed and stores it in the storage unit 39. For example, in response to the operation of an environmental map creation button on the operation display unit 36, the environmental map acquisition unit 60 estimates its own position and creates an environmental map in real time using a technique such as SLAM (Simultaneous Localization and Mapping) while the autonomous traveling cleaning device 4 is traveling in the cleaning area. Note that the autonomous traveling cleaning device 4 may travel manually in response to the operation of the travel operation unit 33 when creating the environmental map, or may simply travel automatically in the cleaning area based on the measurement results of the measurement unit 34.

[0103] At this time, while traveling in the cleaning area, the environmental map acquisition unit 60 acquires position information of the device main body 30 and obstacles around the device main body 30 as the measurement result of the measurement unit 34, and creates a local map of the surroundings of the device main body 30 at predetermined time intervals or predetermined distance intervals based on the measurement result of the measurement unit 34. The environmental map acquisition unit 60 estimates the self-position (coordinates) of the autonomous traveling cleaning device 4 in the local map based on the local map and the detection results (movement amount of the traveling unit 31) by each encoder of the traveling unit 31. The environmental map acquisition unit 60 creates an environmental map of the cleaning area by connecting (combining) each local map.

[0104] Alternatively, the environment map acquisition section 60 may input data of the environment map, such as CAD data, from an external device in response to the operation of an environment map input button on the operation display section .

[0105] The data reflection unit 61 associates the imaging data of the aerial images taken by the unmanned flying device 2 for the cleaning area to be the target of automatic cleaning with the environmental map and stores it in the memory unit 39, and reflects the imaging data on the environmental map. For example, when the environmental map acquisition unit 60 has acquired the environmental map of the cleaning area and imaging data of the aerial images of the cleaning area is received from the unmanned flying device 2 via the communication unit 40, the data reflection unit 61 associates the imaging data with the environmental map.

[0106] The data reflecting unit 61 reflects data on the position (coordinates) and size (range) of non-collected objects acquired from the imaging data by the object discriminating unit 62, and data on the position (coordinates) and size (range) of the collection position set by the collection position setting unit 64 based on the data of non-collected objects, on the environmental map, and similarly superimposes data on the position (coordinates) and size (range) of the collection objects acquired from the imaging data on the environmental map. Alternatively, the data reflecting unit 61 may reflect data on the position (coordinates) and size (range) of non-collected objects acquired from the imaging data by the object discriminating unit 53 in the unmanned flying device 2, and data on the position (coordinates) and size (range) of the collection position set by the collection position setting unit 54, on the environmental map, and superimpose data on the position (coordinates) and size (range) of the collection objects on the environmental map.

[0107] The object discrimination unit 62 discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data of the aerial image of the cleaning area received from the unmanned flying device 2 by the communication unit 40. For example, the object discrimination unit 62 uses an image analysis technique represented by a semantic segmentation technique based on the difference in shape and color between the floor or ground, the objects to be collected such as fallen leaves, and the objects not to be collected such as walls to discriminate between objects to be collected and objects not to be collected and discriminate between objects to be collected and objects not to be collected. The object discrimination unit 62 calculates data on the position (coordinates) and size (range) of the discriminated objects to be collected and objects not to be collected in the cleaning area as discrimination results.

[0108] Furthermore, the object discrimination unit 62 discriminates non-collectible objects on the floor or ground of the cleaning area based not only on the imaging data of the aerial image of the cleaning area but also on the recognition information of obstacles recognized by the obstacle recognition unit 63 based on the measurement results of the measurement unit 34. Note that the object discrimination unit 62 may discriminate between collectible objects and non-collectible objects around the autonomously traveling cleaning device 4 based on surrounding image data obtained by capturing an image of the surroundings of the autonomously traveling cleaning device 4 by the imaging unit 35.

[0109] The obstacle recognition unit 63 recognizes obstacles around the device body 30, for example, recognizes the shape of the obstacles, based on the measurement result obtained by measuring the positional relationship between the device body 30 and the obstacles around the device body 30 by the measurement unit 34. Furthermore, the obstacle recognition unit 63 acquires position information of the obstacle relative to the device body 30. The obstacle recognition unit 63 regards the recognized shape and position information of the obstacle as recognition information. The obstacle recognition unit 63 may recognize obstacles around the autonomously traveling cleaning device 4 based on surrounding image data obtained by capturing an image of the surroundings of the autonomously traveling cleaning device 4 by the imaging unit 35.

[0110] The collection position setting unit 64 determines and sets a collection position for collecting the collection objects in the cleaning area based on the image data of the cleaning area and / or the recognition information of obstacles in the cleaning area, or based on the discrimination result of the collection objects and non-collection objects discriminated by the object discriminator 62. For example, the collection position setting unit 64 sets a collection position on one side of the non-collection objects closer to the autonomously traveling cleaning device 4 among the discriminated non-collection objects.

[0111] Alternatively, the collection position setting unit 64 may determine a concentration position where the objects to be collected are most concentrated, and set a collection position on one side of the objects to be collected that is closest to the concentration position. The collection position setting unit 64 may set one collection position for the cleaning area, or may set multiple collection positions.

[0112] When the data on the non-collectible objects acquired from the image data of the cleaning area by the data reflecting unit 61 is reflected on the environmental map, the cleaning plan creating unit 65 creates a cleaning plan for performing automatic traveling and cleaning to clean garbage such as dirt and sand on the floor or ground of the cleaning area, and stores the plan in association with the cleaning area and the environmental map in the storage unit 39. Furthermore, when the data on the collectible objects is superimposed on the environmental map, the cleaning plan creating unit 65 creates a special cleaning plan for performing automatic traveling and cleaning to clean the collectible objects on the floor or ground of the cleaning area based on the environmental map, and stores the plan in association with the cleaning area and the environmental map in the storage unit 39 separately from the normal cleaning plan. Note that when the collection position is set by the collection position setting unit 64, the cleaning plan creating unit 65 creates a special cleaning plan for performing automatic traveling and cleaning to preferentially clean the collectible objects at the collection position. The cleaning plan creating unit 65 creates a cleaning plan for a travel route that maintains a safe distance from non-collectible objects and obstacles in the environmental map. The cleaning plan creation unit 65 may create a cleaning plan so as to stop cleaning in a location where there is no object to be collected. The cleaning plan creation unit 65 may display the created normal cleaning plan or special cleaning plan on the operation display unit 36 ​​so as to make it editable in response to an operation of the operation display unit 36.

[0113] In response to the operation of the automatic travel and cleaning execution button on the operation display unit 36, the automatic travel and cleaning control unit 66 reads out from the storage unit 39 the environmental map and the cleaning plan or the special cleaning plan associated with the cleaning area to be subjected to the automatic travel and cleaning, and controls the traveling unit 31 and the cleaning unit 32 to perform the automatic travel and cleaning based on the traveling data and cleaning data of the environmental map and the cleaning plan or the special cleaning plan. During the automatic travel and cleaning of the cleaning area with the cleaning plan, the automatic travel and cleaning control unit 66 may change the cleaning plan during the automatic travel and cleaning to the special cleaning plan at the position where the collection object is determined, and continue the automatic travel and cleaning. In other words, the automatic travel and cleaning control unit 66 may switch to and update either the cleaning plan or the special cleaning plan according to the distribution situation of the collection object reflected in the environmental map, and continue the automatic travel and cleaning.

[0114] At this time, the automatic traveling and cleaning control unit 66 performs automatic traveling and cleaning while estimating the self-position of the automatic traveling and cleaning device 4 on the environmental map corresponding to the cleaning plan or the special cleaning plan. For example, the automatic traveling and cleaning control unit 66 uses the environmental map acquisition unit 60 to create a local map using a technology such as SLAM, estimates the self-position of the automatic traveling and cleaning device 4 in the local map, and matches the local map with the environmental map to estimate the self-position on the environmental map. Then, the automatic traveling and cleaning control unit 66 controls the traveling unit 31 by matching the self-position data for each step of the traveling data of the cleaning plan or the special cleaning plan with the self-position estimated on the environmental map, while controlling the cleaning work for each step based on the cleaning data of the cleaning unit 32.

[0115] Furthermore, during automatic traveling cleaning, the automatic traveling cleaning control unit 66 detects non-collectible objects based on the measurement results of the measuring unit 34, and controls the traveling unit 31 to decelerate when the distance between the autonomous traveling cleaning device 4 and the non-collectible object approaches a specified safe distance, and controls the traveling unit 31 to stop when the distance between the autonomous traveling cleaning device 4 and the non-collectible object falls below the specified safe distance.

[0116] The autonomous traveling and cleaning control unit 66 temporarily suspends the autonomous traveling and cleaning when the moving object discrimination unit 67 discriminates a moving object that has moved or is present around the autonomous traveling and cleaning device 4. After that, when the moving object discrimination unit 67 no longer discriminates a moving object, the autonomous traveling and cleaning control unit 66 resumes the autonomous traveling and cleaning.

[0117] The moving object discrimination unit 67 discriminates moving objects such as people that are moving or present around (within a predetermined distance from) the autonomous traveling cleaning device 4. For example, the moving object discrimination unit 67 may be configured with a human sensor or the like to discriminate the moving object, or may discriminate the moving object using an image matching technique based on surrounding image data captured by the imaging unit 35.

[0118] In the cleaning system 1 of this embodiment, the automatic traveling and cleaning operation of the autonomous traveling cleaning device 4 when an operator operates the unmanned flying device 2 to fly it manually will be described with reference to the flowcharts of Figs. 8 and 9.

[0119] First, as shown in FIG. 8, the operator operates the control device 3 to start the unmanned flying device 2 (step S1), and operates the control unit 20 of the control device 3 to control the unmanned flying device 2 to manually fly it above the cleaning area to be cleaned by automatic cleaning. At this time, the control device 3 transmits a flight instruction to the unmanned flying device 2 (step S2), and the flight control unit 50 controls the flying unit 11 of the unmanned flying device 2 in response to the received flight instruction to manually fly the device (step S3). During the manual flight, the image capture control unit 52 controls the image capture unit 12 of the unmanned flying device 2 to capture an aerial image below the unmanned flying device 2, generate image capture data for confirmation, and transmit the image capture data to the control device 3 (step S4). The unmanned flying device 2 may take and transmit a video as the image capture data for confirmation, but it is not limited to a video, and may take a still image every few seconds and transmit it one by one.

[0120] The control device 3 receives the image data for confirmation from the unmanned flying device 2 and displays it on the display unit 21 (step S5), and the worker can control the unmanned flying device 2 while checking the aerial image displayed on the display unit 21. For example, the worker can check the collection target such as fallen leaves on the floor or ground of the cleaning area from the aerial image on the display unit 21 as shown in Figs. 6 and 7, and use the downwash of the flight unit 11 to fly the unmanned flying device 2 so as to blow and collect the collection target toward a predetermined collection direction (collection position) (step S6). In addition, the worker can check the non-collection target such as a wall on the floor or ground of the cleaning area from the aerial image on the display unit 21 as shown in Fig. 7, and control the unmanned flying device 2 to fly so as to blow and collect the collection target with the adjacent position of the non-collection target as the collection position.

[0121] After collecting the objects to be collected at the collection position, while manually flying the unmanned flying device 2 above the cleaning area, the worker operates the image capture transmission button of the control device 3 to cause the unmanned flying device 2 to capture an aerial image of the cleaning area to obtain imaging data for analysis of the cleaning area. At this time, the control device 3 transmits an image capture transmission command to the unmanned flying device 2 (step S7), and in the unmanned flying device 2, the imaging control unit 52 controls the imaging unit 12 in response to the received image capture transmission command to capture an aerial image below the unmanned flying device 2 and generate imaging data (step S8). In addition, the unmanned flying device 2 transmits the captured imaging data for analysis to the autonomously traveling cleaning device 4. The unmanned flying device 2 may capture and transmit at least one still image as imaging data for analysis.

[0122] When the operation of acquiring the imaging data for analyzing the cleaning area is completed, the operator operates the control unit 20 of the control device 3 to control the unmanned flying device 2 and end the manual flight. At this time, the control device 3 transmits a control instruction to end the flight to the unmanned flying device 2 (step S9), and in response to the received control instruction to end the flight, the flight control unit 50 controls the flying unit 11 to end the manual flight (step S10), for example, to return to the starting position of the manual flight. Alternatively, the operator may perform all flights of the unmanned flying device 2 until the flight ends by manual flight by operating the control device 3. The operator may continue to hover the unmanned flying device 2 without ending the flight.

[0123] On the other hand, as shown in FIG. 9, after the autonomous traveling cleaning device 4 is started (step S11), the environmental map acquisition unit 60 acquires an environmental map of the cleaning area where the autonomous traveling cleaning is performed, and stores it in the storage unit 39 (step S12). In addition, the autonomous traveling cleaning device 4 receives imaging data for analyzing the cleaning area captured by the unmanned flying device 2 from the unmanned flying device 2 (step S13), and performs image analysis on the imaging data using the object discrimination unit 62 to acquire data on the position (coordinates) and size (range) of non-collectible objects and data on the position (coordinates) and size (range) of collectible objects (step S14). Furthermore, the data reflection unit 61 reflects the acquired data of non-collectible objects on the environmental map, and superimposes the acquired data of collectible objects on the environmental map (step S15).

[0124] The autonomously moving cleaning device 4 uses the object discrimination unit 62 to discriminate between collection objects and non-collection objects on the floor or ground of the cleaning area based on data on collection objects and non-collection objects and collection locations obtained from the imaging data received from the unmanned flying device 2 (step S16).

[0125] If there are no objects to be collected in the cleaning area (step S16: No), the autonomously traveling cleaning device 4 creates a normal cleaning plan for performing autonomous traveling cleaning to clean up debris such as dirt and sand using the cleaning plan creation unit 65 (step S17), and stores the plan in the memory unit 39 in association with the cleaning area and environmental map.

[0126] On the other hand, if there are collection objects in the cleaning area (Step S16: Yes), the autonomously traveling cleaning device 4 creates a special cleaning plan for performing autonomous traveling cleaning so as to prioritize and intensively clean the collection objects on the floor or ground of the cleaning area based on the environmental map on which the data of non-collection objects in the cleaning area is reflected and on which the data of the collection objects is superimposed (Step S18), and stores the plan in the memory unit 39 in association with the cleaning area and the environmental map.

[0127] Next, the autonomous traveling cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 to perform autonomous traveling cleaning by the autonomous traveling cleaning control unit 66 based on the environmental map and the cleaning plan or special cleaning plan associated with the cleaning area to be subjected to autonomous traveling cleaning (step S19).

[0128] In the cleaning system 1 of this embodiment, the automatic traveling and cleaning operation of the autonomous traveling cleaning device 4 when the unmanned flying device 2 is caused to fly automatically will be described with reference to the flowchart of FIG.

[0129] First, the operator operates the control device 3 to start the unmanned flying device 2 (step S21), and operates the automatic flight button of the control device 3 to automatically fly the unmanned flying device 2 in the cleaning area to be subjected to automatic cleaning (step S22). At this time, the control device 3 transmits an automatic flight instruction to the unmanned flying device 2, and the unmanned flying device 2 starts automatic flight in response to the received automatic flight instruction, and first, the flight control unit 50 controls the flying unit 11 based on the preliminary flight plan to perform a pre-flight. During the pre-flight, the imaging control unit 52 of the unmanned flying device 2 controls the imaging unit 12 to capture an aerial image of the cleaning area and generate imaging data for analysis of the cleaning area. That is, imaging data of the cleaning area before the collection of the collection object is generated (step S23).

[0130] When the imaging data of the cleaning area is generated, the object discrimination unit 53 of the unmanned flying device 2 performs image analysis on the generated imaging data (step S23), acquires data of non-collectible objects on the floor or ground of the cleaning area, and transmits the data to the autonomously traveling cleaning device 4 (step S24). Furthermore, the object discrimination unit 53 acquires data of the collectible objects on the floor or ground of the cleaning area by performing image analysis on the generated imaging data (step S25). After that, the object discrimination unit 53 discriminates the collectible objects and non-collectible objects on the floor or ground of the cleaning area based on the imaging data in order to create a collection flight plan by the flight plan creation unit 51. In addition, the collection position setting unit 54 of the unmanned flying device 2 determines and sets the collection position where the collectible objects should be collected in the cleaning area based on the acquired data of the collectible objects and non-collectible objects (step S26), as shown in FIG. 14.

[0131] When data on the objects to be collected and the objects not to be collected are acquired by image analysis of the image data of the cleaning area, the flight plan creation unit 51 of the unmanned flying device 2 creates a collection flight plan based on the identified objects to be collected and the objects not to be collected and the set collection positions (step S27). At this time, the flight plan creation unit 51 creates a collection flight plan for flying in a manner that avoids the objects not to be collected and blows the objects to be collected by downwash to the collection positions.

[0132] When the collection flight plan is created, the flight control unit 50 controls the flight unit 11 based on the collection flight plan to perform an automatic collection flight (step S28), as shown in Fig. 14. By this automatic collection flight, the unmanned flying device 2 flies to perform a collection auxiliary operation of blowing and gathering the collection objects on the floor or ground of the cleaning area toward the collection position, using the downwash of the flight unit 11, as shown in Figs. 14 and 15.

[0133] At the end of the automatic collection flight, the flight control unit 50 controls the flight unit 11 based on the collection flight plan to perform a post-flight, and the imaging control unit 52 controls the imaging unit 12 to capture an aerial image of the cleaning area and generate imaging data for analyzing the cleaning area. That is, imaging data of the cleaning area after collection of the collection objects is generated, and the object discrimination unit 53 performs image analysis on the generated imaging data (step S29). The unmanned flying device 2 acquires data of the collection objects in the imaging data from the results of the image analysis, and transmits the data of the collection objects in the imaging data together with data of the collection position that has already been set to the autonomously traveling cleaning device 4 (step S30).

[0134] When the acquired data of the collection target and the data of the collection position already set are transmitted to the autonomously traveling cleaning device 4, the unmanned flying device 2 ends the automatic flight. The flight control unit 50 controls the flying unit 11 to end the automatic flight, for example, to return to the starting position of the automatic flight. Alternatively, the flight control unit 50 may continue to hover the unmanned flying device 2 over the cleaning area.

[0135] On the other hand, after the autonomous traveling cleaning device 4 is started (step S31), the environmental map acquisition unit 60 acquires an environmental map of the cleaning area where the autonomous traveling cleaning is performed, and stores the map in the storage unit 39 (step S32). The environmental map acquisition unit 60 creates an environmental map according to the measurement results of the LRF 34a of the measurement unit 34, for example. At this time, as shown in FIG. 12, the measurement range of the LRF 34a in the cleaning area is shown as a sector around the autonomous traveling cleaning device 4 (LRF 34a), but randomly parked vehicles may become obstacles as seen from the LRF 34a, blocking the measurement range, causing blind spots in the measurement of the LRF 34a, and non-collection targets that cannot be detected from the measurement results of the measurement unit 34 may exist.

[0136] In addition, the autonomous traveling cleaning device 4 receives data on non-collectible objects in the cleaning area acquired by the unmanned flying device 2 from the unmanned flying device 2, and reflects the data on the non-collectible objects on the environmental map by the data reflecting unit 61 (step S33). At this time, the imaging range of the unmanned flying device 2 is wide as shown by the dashed line in FIG. 13, and the imaging data of the cleaning area to be imaged includes a range that cannot be grasped by the measurement results of the measuring unit 34. Therefore, by reflecting the non-collectible objects acquired by image analysis of the imaging data on the environmental map, it is possible to reflect the existence of the cleaning area that cannot be detected from the measurement results of the measuring unit 34 due to the blind spot of an obstacle such as a vehicle on the environmental map. After that, it is sufficient to reflect the collection object and the collection position. By reflecting the entire cleaning area without blind spots on the environmental map, it is possible to reliably clean the collection object that exists in the blind spot.

[0137] The autonomous cleaning device 4 superimposes the collection objects on the floor or ground of the cleaning area onto an environmental map based on the data of the collection objects and collection locations received from the unmanned flying device 2 using the data reflection unit 61, and reflects the collection locations on the environmental map (step S35).

[0138] If there are no objects to be collected in the cleaning area (step S36: No), the autonomously traveling cleaning device 4 creates a normal cleaning plan for performing autonomous traveling cleaning to clean up debris such as dirt and sand using the cleaning plan creation unit 65 (step S37), and stores the plan in the memory unit 39 in association with the cleaning area and environmental map.

[0139] On the other hand, if there are objects to be collected in the cleaning area (Step S36: Yes), the autonomously traveling cleaning device 4 creates a special cleaning plan for performing autonomous traveling cleaning to clean the objects to be collected on the floor or ground in the cleaning area, as shown in FIG. 15, based on the environmental map on which the data of the objects to be collected and the data of the objects to be collected in the cleaning area is reflected by the cleaning plan creation unit 65 and superimposed (Step S38), and stores the plan in the memory unit 39 in association with the cleaning area and the environmental map.

[0140] Next, the autonomous traveling cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 to perform autonomous traveling cleaning by the autonomous traveling cleaning control unit 66 based on the environmental map and the cleaning plan or special cleaning plan associated with the cleaning area to be subjected to the autonomous traveling cleaning (step S39).

[0141] In the configuration in which the unmanned flying device 2 is equipped with the object discrimination unit 53 and collection position setting unit 54 as described above, a large computing capacity is required for the control unit 16 of the unmanned flying device 2, and the unmanned flying device 2 that can be implemented is limited. Therefore, the operation of setting collection objects, non-collection objects, and collection positions in imaging data using the object discrimination unit 62, obstacle recognition unit 63, and collection position setting unit 64 of the autonomously traveling cleaning device 4 will be described with reference to the flowchart of FIG.

[0142] The worker operates the control device 3 to start the unmanned flying device 2 (step S41), and operates the automatic flight button of the control device 3 to automatically fly the unmanned flying device 2 in the cleaning area to be subjected to automatic traveling and cleaning (step S42). At this time, the control device 3 transmits an automatic flight instruction to the unmanned flying device 2, and the unmanned flying device 2 starts a preliminary automatic flight in response to the received automatic flight instruction. During the automatic flight, the unmanned flying device 2 has the imaging control unit 52 control the imaging unit 12 to capture an aerial image of the cleaning area, generate imaging data for analyzing the cleaning area, and transmit it to the autonomously traveling cleaning device 4 (step S43).

[0143] On the other hand, the autonomous traveling cleaning device 4, after starting (step S51), similarly to the above, acquires an environmental map of the cleaning area where the autonomous traveling cleaning is performed by the environmental map acquisition unit 60 and stores it in the storage unit 39 (step S52). In addition, when the autonomous traveling cleaning device 4 receives imaging data from the unmanned flying device 2 (step S53), the object discrimination unit 62 performs image analysis based on the imaging data, acquires and discriminates data of the collection object and the non-collection object on the floor surface or ground of the cleaning area (step S54). On the other hand, the data of the non-collection object is reflected in the environmental map by the data reflection unit 61 (step S55), and the cleaning plan creation unit 65 creates a normal cleaning plan in advance for performing the autonomous traveling cleaning so as to clean up garbage such as soil and sand in the cleaning area (step S56). Thereafter, based on the data of the non-collection object and the collection object acquired and discriminated from the result of the image analysis, the collection position setting unit 64 sets a collection position where the collection object should be collected in the cleaning area, as shown in FIG. 14. The autonomously traveling cleaning device 4 transmits data on the identified non-collection targets and collection targets, as well as data on the set collection position, to the unmanned flying device 2 (step S57).

[0144] When the unmanned flying device 2 receives data on the non-collectible objects and the collectible objects and the set collection location (step S44), the flight plan creation unit 51 creates a collection flight plan based on the determined collectible objects and non-collectible objects and the set collection location (step S45). The flight control unit 50 of the unmanned flying device 2 controls the flight unit 11 based on the created collection flight plan to perform an automatic collection flight, and collects the collectible objects toward the collection location (step S46). At the end of the automatic collection flight, the flight control unit 50 controls the flight unit 11 based on the collection flight plan to perform a post-flight, and the imaging control unit 52 controls the imaging unit 12 to capture an aerial image of the cleaning area and generate imaging data for analysis of the cleaning area. That is, imaging data of the cleaning area after collection of the collection objects is generated and transmitted to the autonomously traveling cleaning device 4 (step S47).

[0145] When the autonomous traveling cleaning device 4 receives the imaging data for analysis from the unmanned flying device 2 that performed the automatic collection flight, the object discrimination unit 62 performs image analysis based on the imaging data to acquire and discriminate data on the objects to be collected on the floor or ground of the cleaning area (step S58). If the image analysis result shows that there is no object to be collected in the received imaging data (step S59: No), the automatic traveling and cleaning control unit 66 controls the traveling unit 31 and the cleaning unit 32 to perform automatic traveling and cleaning based on the environmental map associated with the cleaning area and the traveling data and cleaning data of the normal cleaning plan for performing automatic traveling and cleaning to clean up dirt such as soil and sand that has already been created by the cleaning plan creation unit 65 (step S62).

[0146] On the other hand, if the image analysis result indicates that the collected object is present in the received image data (step S59: Yes), the data reflecting unit 61 superimposes the acquired data of the collected object on the environmental map, reflects and sets the data of the collection position on the environmental map (step S60), and the cleaning plan creating unit 65 creates a special cleaning plan for performing automatic traveling and cleaning so as to clean the collected object on the floor or ground of the cleaning area, and stores the special cleaning plan in association with the cleaning area and the environmental map in the storage unit 39. After that, the cleaning plan creating unit 65 updates the normal cleaning plan for automatic traveling and cleaning the cleaning area to the special cleaning plan (step S61). Next, the autonomous traveling and cleaning device 4 controls the traveling unit 31 and the cleaning unit 32 to perform automatic traveling and cleaning based on the environmental map and the special cleaning plan associated with the cleaning area to be subjected to automatic traveling and cleaning by the autonomous traveling and cleaning control unit 66 (step S62).

[0147] For example, the autonomous traveling cleaning device 4 may perform autonomous traveling cleaning immediately after creating a cleaning plan in advance by reflecting data on non-collectible objects acquired by image analysis of the imaging data transmitted from the above-mentioned unmanned flying device 2 in the environmental map in advance. Furthermore, when the autonomous traveling cleaning device 4 determines data on collectible objects by image analysis of the imaging data transmitted from the unmanned flying device 2 while performing autonomous traveling cleaning according to a normal cleaning plan, it may create a special cleaning plan based on the data on the collectible objects, update the cleaning plan to the special cleaning plan, and continue autonomous traveling cleaning.

[0148] As described above, according to the present embodiment, the cleaning system 1 including the autonomously traveling cleaning device 4 capable of performing automatic traveling cleaning, which autonomously travels and automatically cleans, further includes the unmanned flying device 2. The autonomously traveling cleaning device 4 includes a device main body 30, a traveling unit 31 that travels the device main body 30, a cleaning unit 32 that cleans on the travel path of the device main body 30, a control unit 38 that functions as an environmental map acquisition unit 60 that acquires an environmental map of the cleaning area where the automatic traveling cleaning is performed, and a communication unit 40 (first communication unit) that wirelessly communicates with the unmanned flying device 2. The unmanned flying device 2 includes an imaging unit 12 and a communication unit 18 (second communication unit) that wirelessly communicates with the autonomously traveling cleaning device 4. When the unmanned flying device 2 flies above the cleaning area, the imaging unit 12 images the cleaning area from the sky and generates imaging data. The communication unit 18 wirelessly communicates with the communication unit 40 and transmits the imaging data to the autonomously traveling cleaning device 4. The control unit 38 of the autonomous traveling cleaning device 4 functions as a data reflection unit 61 that reflects the imaging data captured by the imaging unit 12 on the environmental map by associating the imaging data with the environmental map.

[0149] With this configuration, the cleaning system 1 can obtain information on blind spots that the autonomous cleaning device 4 cannot see by reflecting the image data of the cleaning area captured by the unmanned flying device 2 on the environmental map, and can obtain an environmental map showing the cleaning area in more detail. Therefore, the autonomous cleaning device 4 can recognize the entire cleaning area.

[0150] In this embodiment, the control unit 38 of the autonomous traveling cleaning device 4 functions as an object discrimination unit 62 that discriminates between objects to be collected and objects to be not collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit 12. The data reflection unit 61 reflects the objects to be collected and objects to be not collected that are discriminated by the object discrimination unit 62 on the environmental map. The control unit 38 of the autonomous traveling cleaning device 4 functions as a cleaning plan creation unit 65 that creates a cleaning plan for performing autonomous traveling cleaning so as to clean objects to be collected based on the environmental map, and functions as an autonomous traveling cleaning control unit 66 that performs autonomous traveling cleaning based on the cleaning plan.

[0151] This makes it possible to obtain information about blind spots that the autonomous cleaning device 4 cannot see, and to create a cleaning plan that ensures more reliable collection so that no items to be collected are left behind in the cleaning area.

[0152] In this embodiment, the unmanned flying device 2 is equipped with a control unit 16 that functions as a flight control unit 50 that controls the collection operation of blowing and collecting the collection objects to a predetermined collection position within the cleaning area by utilizing downwash during flight. After performing the collection operation, the imaging unit 12 captures an image of the cleaning area including the collection position to generate imaging data. The data reflection unit 61 reflects the collection position captured in the imaging data on the environmental map. The cleaning plan creation unit 65 creates a cleaning plan for performing automatic traveling cleaning so as to preferentially clean the collection objects at the collection position based on the environmental map.

[0153] As a result, the autonomous cleaning device 4 can clean most of the objects in the cleaning area by cleaning the collection location where the objects were collected, improving cleaning efficiency. Therefore, objects in the cleaning area can be collected more easily and reliably so that none are left behind.

[0154] In this embodiment, the control unit 38 of the autonomous traveling cleaning device 4 functions as an obstacle recognition unit 63 that recognizes obstacles based on the measurement results of the positional relationship of obstacles around the device body 30. The object discrimination unit 62 discriminates non-collection objects on the floor or ground of the cleaning area based on the recognition information of the obstacles recognized by the obstacle recognition unit 63. The control unit 38 of the autonomous traveling cleaning device 4 also functions as a collection position setting unit 64 that sets a collection position on one side of the non-collection object discriminated by the object discrimination unit 62 based on the imaging data or recognition information.

[0155] This allows the non-collection objects to be used at the collection position, so that the objects to be collected can be blown together more reliably, and the objects to be collected can be collected and cleaned more easily and reliably so that no objects to be collected are left behind in the cleaning area.

[0156] In other words, in this embodiment, the unmanned flying device 2 includes an imaging unit 12 that captures an aerial image of the cleaning area from above when flying over the cleaning area to generate imaging data, a control unit 16, and a communication unit 18 that performs wireless communication with the autonomously traveling cleaning device 4. The control unit 16 functions as an object discrimination unit 53 that discriminates between objects to be collected and objects to be not collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit 12, functions as a collection position setting unit 54 that sets a collection position on one side of the objects to be not collected that are discriminated based on the imaging data by the object discrimination unit 53, and functions as a flight control unit 50 that controls the collection operation of blowing and collecting objects to a collection position in the cleaning area by using downwash during flight. After the collection operation is performed, the imaging unit 12 captures the cleaning area including the collection position to generate imaging data. The communication unit 18 transmits the imaging data captured by the imaging unit 12 to the autonomously traveling cleaning device 4 through wireless communication with the autonomously traveling cleaning device 4 after the collection operation is performed.

[0157] This allows the unmanned flying device 2 to provide the autonomously traveling cleaning device 4 with the collection location where the objects to be collected were collected, so that the autonomously traveling cleaning device 4 can clean most of the objects to be collected in the cleaning area by cleaning this collection location, improving cleaning efficiency. This allows objects to be collected more easily and reliably in the cleaning area, without leaving any objects behind.

[0158] In this embodiment, the unmanned flying device 2 further includes a cleaning device detection unit 55 that detects the position of the autonomously traveling cleaning device 4 based on the imaging data. The collection position setting unit 54 sets a collection position on one side of the non-collection objects that are close to the autonomously traveling cleaning device 4 among the non-collection objects identified by the object discrimination unit 53.

[0159] This allows non-collection objects close to the autonomous cleaning device 4 to be used as collection locations, so that the objects can be more reliably blown together and collected, making it easier and more reliable to collect objects in the cleaning area without leaving any behind.

[0160] Furthermore, in this embodiment, the cleaning device detection unit 55 detects the beacon signal sent out by the autonomous traveling cleaning device 4 to detect the position of the autonomous traveling cleaning device 4.

[0161] For example, the cleaning device detection unit 55 detects the position of the autonomously traveling cleaning device 4 by detecting, as a beacon signal, infrared rays transmitted by a laser range finder in order for the autonomously traveling cleaning device 4 to measure the positional relationship of surrounding obstacles.

[0162] This allows the position of the autonomously traveling cleaning device 4 to be grasped more easily and reliably, and non-collection objects close to the autonomously traveling cleaning device 4 can be used as collection locations for collection objects, making it possible to more reliably gather the collection objects.

[0163] In addition, in this embodiment, the unmanned flying device 2 is equipped with a moving object discrimination unit 56 that discriminates moving objects moving on the floor or ground of the cleaning area, and the flight control unit 50 interrupts the flight when a moving object is discriminated by the moving object discrimination unit 56.

[0164] This allows the unmanned flying device 2 to improve safety by interrupting flight if a moving object such as a person is present when flying to take aerial images of a cleaning area or to collect collection items.

[0165] In the above embodiment, an example has been described in which, when there is a non-collection target in the cleaning area, the collection position setting unit 54 of the unmanned flying device 2 and the collection position setting unit 64 of the autonomously traveling cleaning device 4 set a collection position on one side of the non-collection target, but the present invention is not limited to this example. For example, as shown in Fig. 16, when there is no non-collection target in the cleaning area, the collection position setting unit 54 and the collection position setting unit 64 cannot determine the non-collection target for which to set a collection position.

[0166] 17, the worker may set up landmarks such as folding screens or partitions that separate the cleaning area on the floor or ground of the cleaning area, and the collection position setting unit 54 and the collection position setting unit 64 may recognize the landmarks based on the imaging data of the cleaning area captured by the imaging unit 12, and set the collection position on one side of the recognized landmark. Note that the landmarks may be configured to have a repeating shape so that they can be identified from the imaging data captured by the imaging unit 12 of the unmanned flying device 2 and the measurement results measured by the measurement unit 34 of the autonomously traveling cleaning device 4.

[0167] For example, if the object discrimination unit 53 of the unmanned flying device 2 or the object discrimination unit 62 of the autonomously moving cleaning device 4 does not identify any non-collectable objects based on the image data of the cleaning area, the unmanned flying device 2, the piloting device 3 or the autonomously moving cleaning device 4 may notify the worker that there are no non-collectable objects and prompt them to install a landmark.

[0168] This allows a landmark for blowing and collecting the collection objects to be easily set up in the cleaning area even if there are no non-collection objects in the cleaning area onto which the collection objects can be blown and collected, making it possible to blow and collect the collection objects.

[0169] Furthermore, the present invention can be modified as appropriate within the scope that does not contradict the gist or concept of the invention that can be read from the claims and the entire specification, and cleaning systems, unmanned flying devices, and autonomous cleaning devices that involve such modifications are also included in the technical concept of the present invention. [Industrial Applicability]

[0170] The present invention can be suitably used for a cleaning robot that uses an autonomous traveling cleaning device capable of performing autonomous traveling and automatic cleaning to clean a wide area such as a park or parking lot as a cleaning area, and travels through the cleaning area to collect and clean garbage such as dust and fallen leaves scattered on the floor or ground of the cleaning area as collection objects. Also, the present invention can be suitably used for a cleaning system that includes an autonomous traveling cleaning device such as a cleaning robot that performs autonomous traveling and cleaning, and an unmanned flying device such as a drone equipped with an imaging unit such as a camera. [Explanation of symbols]

[0171] 1. Cleaning System 2 Unmanned flying device 3. Controls 4. Autonomous Cleaning Device 11. Flying Club 12 Imaging unit 16 Control section 18 Communication Department (Second Communication Department) 31 Running part 32 Cleaning Department 34 Measurement section 38 Control Unit 40 Communication Department (First Communication Department) 50 Flight control unit 51 Flight Planning Department 52 Imaging control section 53 Object discrimination unit 54 Collection position setting section 55 Cleaning device detector 56 Moving object discrimination unit 60 Environmental Map Acquisition Department 61 Data Reflection Department 62 Object discrimination unit 63 Obstacle Recognition Unit 64 Collection position setting section 65 Cleaning Plan Creation Department 66 Automatic travel and cleaning control unit 67 Moving object discrimination unit

Claims

1. A cleaning system including an autonomous cleaning device capable of performing autonomous cleaning by autonomously traveling and automatically cleaning, Further comprising an unmanned flying device, The autonomous cleaning device includes: A device body, A running unit that runs the device body; a cleaning unit that performs cleaning on a travel path of the device body; an environmental map acquisition unit that acquires an environmental map of a cleaning area in which the automatic cleaning is performed; A first communication unit that wirelessly communicates with the unmanned flying device, The unmanned flying device is An imaging unit; a second communication unit that wirelessly communicates with the autonomously traveling cleaning device; The imaging unit captures an image of the cleaning area from above when the unmanned flying device flies above the cleaning area to generate imaging data, The second communication unit wirelessly communicates with the first communication unit to transmit the imaging data to the autonomous traveling cleaning device, The autonomous cleaning device and / or the unmanned flying device includes an object discrimination unit that discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit, A cleaning system characterized in that the autonomously traveling cleaning device is equipped with a data reflection unit that reflects the imaging data in the environmental map by associating the collection objects and non-collection objects identified by the object discrimination unit with the environmental map.

2. The autonomous cleaning device includes: a cleaning plan creation unit that creates a cleaning plan for performing the automatic traveling cleaning so as to clean the collection objects based on the environmental map; The cleaning system according to claim 1 , further comprising: an automatic traveling and cleaning control unit that executes the automatic traveling and cleaning based on the cleaning plan.

3. The unmanned flying device is equipped with a flight control unit that controls a collection operation to blow and collect the collection objects to a predetermined collection position within the cleaning area by utilizing downwash during flight, The imaging unit, after the collection operation is performed, captures an image of the cleaning area including the collection position to generate the imaging data; The data reflecting unit reflects the collection position captured in the imaging data on the environmental map, The cleaning system according to claim 2, characterized in that the cleaning plan creation unit creates the cleaning plan for performing the automatic traveling cleaning so as to give priority to cleaning the collection objects at the collection location based on the environmental map.

4. The autonomous traveling cleaning device includes an obstacle recognition unit that recognizes an obstacle based on a measurement result of a positional relationship of the obstacle around the device body, The object discrimination unit discriminates the non-collection objects on a floor surface or a ground surface of the cleaning area based on the recognition information of the obstacle recognized by the obstacle recognition unit, The cleaning system according to claim 3 , further comprising a collection position setting unit that sets the collection position to one side of the non-collection object determined by the object determination unit based on the imaging data or the recognition information.

5. An unmanned flying device, an imaging unit that captures an image of a cleaning area from above when flying over the cleaning area to generate imaging data; an object discrimination unit that discriminates between objects to be collected and objects not to be collected on the floor or ground of the cleaning area based on the imaging data captured by the imaging unit; a collection position setting unit that sets a collection position on one side of the non-collection object determined by the object determination unit based on the imaging data; A flight control unit that controls a collection operation to blow and collect the collection objects to the collection position within the cleaning area by using downwash during flight; A communication unit that wirelessly communicates with the autonomous cleaning device; Equipped with The imaging unit, after the collection operation is performed, captures an image of the cleaning area including the collection position to generate the imaging data; An unmanned flying device characterized in that the communication unit transmits the imaging data captured by the imaging unit after the collection operation is performed to the autonomously traveling cleaning device via wireless communication with the autonomously traveling cleaning device.

6. a cleaning device detection unit that detects a position of the autonomously traveling cleaning device based on the imaging data; The unmanned flying device described in claim 5, characterized in that the collection position setting unit sets the collection position on one side of the non-collection objects identified by the object discrimination unit that are closer to the autonomously moving cleaning device.

7. The unmanned flying device described in claim 5, characterized in that the collection position setting unit is erected on the floor or ground of the cleaning area, recognizes landmarks that separate the cleaning area based on the imaging data, and sets the collection position on one side of the recognized landmark.

8. The unmanned flying device according to claim 6, wherein the cleaning device detection unit detects a beacon signal transmitted by the autonomously traveling cleaning device to detect the position of the autonomously traveling cleaning device.

9. The unmanned flying device of claim 8, characterized in that the cleaning device detection unit detects the position of the autonomously traveling cleaning device by detecting infrared rays emitted by a laser range finder as a beacon signal to measure the positional relationship of surrounding obstacles.

10. a moving object discrimination unit that discriminates a moving object moving on a floor surface or a ground surface of the cleaning area, The unmanned flying device according to claim 5, wherein the flight control unit suspends flight when the moving object determination unit determines that the moving object is a moving object.

11. An autonomous traveling cleaning device capable of performing autonomous traveling and cleaning, A device body, A running unit that runs the device body; a cleaning unit that performs cleaning on a travel path of the device body; an environmental map acquisition unit that acquires an environmental map of a cleaning area in which the automatic cleaning is performed; A communication unit that receives imaging data generated by an unmanned flying device flying above the cleaning area and imaging the cleaning area from above by wireless communication with the unmanned flying device; An object discrimination unit that discriminates collection objects and non-collection objects on the floor or ground of the cleaning area based on the imaging data received from the unmanned flying device by the communication unit; An autonomous cleaning device comprising: a data reflection unit that reflects the imaging data in the environmental map by associating the collection objects and non-collection objects identified by the object discrimination unit with the environmental map.

Images