Method for bypassing impassable obstacles by a robot

Abstract

The invention relates to a method for bypassing impassable obstacles by a robot (2) using artificial intelligence (16). Reliable and cost-effective bypassing of obstacles is achieved in consideration of data protection law by generating an optical original recording of the obstacle in the event of a collision of the robot (2) with the obstacle, wherein a copy (25) is artificially generated from the original recording, said copy being used to train the artificial intelligence (16). The invention also relates to a system (1) having a robot (2) and an IT infrastructure (3) designed to carry out the method. The invention also relates to a corresponding computer program and a computer-readable medium having the computer program.

Description

TECHNICAL FIELD

[0001] The invention relates to a method for bypassing impassable obstacles by a robot, in particular a domestic robot, using artificial intelligence. The invention also relates to a system comprising a robot and an IT infrastructure and designed for carrying out the method. Furthermore, the invention relates to a corresponding computer program and a corresponding computer-readable medium. BACKGROUND

[0002] Robots, in particular domestic robots, move through rooms in operation. Domestic robots move through predefined rooms, for example, in a household. For this purpose, the robot is usually provided with a routine for moving, also referred to below as a movement routine, which is carried out taking into account the surroundings of the robot. Important in such routines is the recognition and bypassing of obstacles that the robot cannot pass through.

[0003] Methods are known from the prior art for implementing and / or improving the routines of domestic robots using artificial intelligence.

[0004] For example, DE 10 2016 124 916 A1 discloses a cleaning robot as a domestic robot, which generates obstacle images described manually by a user by means of optical recording devices. The artificial intelligence uses the descriptions associated with the respective images in order to implement the recognition of the respective obstacle in the movement routine in such a way that the obstacle is bypassed. A particular disadvantage here is the need for the user to describe the respective images manually.

[0005] US 2018 / 0 210 445 A1 discloses a domestic robot, which has a sensor device for recognizing obstacles. When an obstacle is recognized, an image of the obstacle is generated and stored in a room map set for the movement routine through which the domestic robot moves. Artificial intelligence is used to recognize properties of the obstacle. The knowledge of the properties of the obstacle obtained by the artificial intelligence is then used in the movement routine of the robot. A disadvantage here is that when an obstacle is displaced in the room, this obstacle is not recognized or is recognized with low reliability.

[0006] It is therefore generally known from the prior art that known obstacles are taken into account in the movement routine in such a way that they are bypassed. Since the number and frequency of potential obstacles varies from room to room and can change over time and also in terms of location, it is desirable to be able to reliably recognize and bypass new or unknown obstacles in the movement routine.

[0007] An improvement in the area of one aspect, namely the mobile routine, and in connection therewith in the area of the use of artificial intelligence is so-called machine learning. For this purpose, so-called training data are required, with which the artificial intelligence is trained in order to improve the mobile routine. Thus, in the mobile routine of an optical detection of the surroundings of a robot, a large number of optical recordings are required for the machine learning. Due to the described plurality of possible obstacles and the change in their position, a large number of optical recordings are required in order to optimise the artificial intelligence accordingly. The problem here is that such a large number of recordings does not exist and / or the acquisition of such recordings is cost-intensive. Furthermore, the consideration of data protection principles leads to the fact that the required recordings cannot be generated and used at will. SUMMARY

[0008] The task to which the present application relates is therefore to specify improved embodiments or at least different embodiments for a method and a related system for bypassing obstacles by a robot, which are distinguished in particular in that the obstacles are bypassed in a low-cost and / or efficient and / or data protection law-compliant manner.

[0009] According to the application, this object is solved by the subject matter of the independent claims. Advantageous embodiments are the subject matter of the dependent claims.

[0010] The present application is based on the common idea that, in order to bypass an impassable obstacle by a robot, the respective obstacle is recognised by a collision of the robot with the obstacle, an optical recording of the obstacle is generated and a large number of copies of one another are artificially generated from the optical recording for training an artificial intelligence, wherein the training result is then used such that the robot bypasses the obstacle in the future. It is in particular involved here that the obstacle is unknown at the point in time of the collision for the mobile routine used by the mobile robot. The solution according to the present application thus makes it possible to train an artificial intelligence by generating an optical recording, also referred to below as an original recording, of an obstacle which is in particular unknown or new, in such a way that a large number of copies are artificially generated from the original recording. Thus, an obstacle which is unknown until the collision can in particular also be taken into account in the mobile routine in such a way that the obstacle is bypassed, i.e. a collision with the obstacle is prevented. At the same time, the number of original recordings required for training the artificial intelligence is reduced, in particular only a single original recording of the respective obstacle is required. Thus, the training of the artificial intelligence and the improvement of the mobile routine can be achieved on the basis of the reduced number of original recordings, in particular on the basis of a single original recording of the obstacle, in a low-cost and reliable manner and taking into account the provisions of data protection law. Furthermore, the idea according to the present application leads to the fact that the respective surroundings, in particular the respective room through which the robot moves, can be reliably bypassed, in particular even in the case of a change in the position of a specific obstacle in the room.

[0011] According to the idea of the present application, in the method for bypassing impassable obstacles by a robot, a collision of the robot with an obstacle is recognized when the robot moves through a room. When the collision is recognized, an optical original recording, in particular a photo, of the obstacle is generated. A large number of copies is then generated from the original recording by hand, wherein the copies each take account of the geometry of the obstacle and differ from one another. These copies are then used at least partially in a training process for training an artificial intelligence, in particular a neural network. This means that the artificial intelligence, in particular the neural network, is trained with at least a part of the copies in such a way that the robot recognizes the obstacle before colliding with the obstacle. The result of this training process, also referred to below as the training result, is subsequently used for bypassing the obstacle by the robot.

[0012] The method according to the present application is thus triggered by a collision of the robot with an obstacle, wherein the collision simultaneously serves as an identification which identifies the obstacle as an impassable obstacle; or at least serves as an indication for an impassable obstacle.

[0013] The training result is expediently used for bypassing the obstacle by taking account of and / or integrating the training result in a movement routine.

[0014] The method according to the present application is in particular a computer-implemented method. The method can thus in particular be executed by means for data processing.

[0015] In principle, the robot can be any robot which moves through a room in operation.

[0016] The robot is in particular a domestic robot which moves through a home in operation. In this case, the robot can be used for cleaning purposes. The robot can thus for example be a cleaning robot, for example a vacuum cleaning robot.

[0017] In a preferred embodiment, the original record is also used to generate copies for training. This means that a large number of copies is generated from the original record, which take account of the geometry of the obstacle and differ from one another, wherein these copies are used in the training process. These copies are also referred to as training copies in the following. Furthermore, a large number of copies is generated from the original record, which take account of the geometry of the obstacle and differ from one another, wherein these copies are used to test the training process. These copies are also referred to as test copies in the following. The artificial intelligence, in particular the neural network, is trained in the training process using the training copies. The training process is interrupted here intermittently, and the intermediate results of the training process are tested in the testing process using at least a part of the test copies. In the respective testing process, the intermediate results are tested in such a way that the robot recognizes the obstacle before the collision with what probability. If this probability is above a predefined value, the intermediate result is set and used as a training result. Conversely, if this probability is below a predefined value, the training process continues and is then again tested in the interval by the testing process. Thus, on the basis of the original record, in particular a single original record, of an obstacle, both the training of the artificial intelligence and the testing of the artificial intelligence, in particular the testing of the training results, are achieved.

[0018] In generating the artificial copies, the obstacle geometry originating from the original record is taken into account in each case. This means, inter alia, that the geometry of the obstacle is retained in at least a part of the copies.

[0019] Alternatively or additionally, the geometry of the obstacle can be changed slightly in at least a part of the copies. This slight change is possible in particular if the artificial intelligence is able to assign the obstacle to a known object, wherein individual features of the object can be altered. By way of example, a door is mentioned here as an obstacle. If the obstacle is identified as a door, for example, the shape and / or size of the door handle can be altered to create different copies. Likewise, the relative arrangement of the handles can be altered to generate different copies.

[0020] Advantageously, at least a part of the copies is generated in such a way that the surroundings, in particular the background, of the obstacle differ from one another in the copies. That is to say, the surroundings, in particular the background, of the obstacle are artificially varied to create different copies.

[0021] Alternatively or additionally, it is conceivable that at least a part of the copies is generated in such a way that the position of the obstacle in the copies differs from one another. That is to say, the position of the obstacle is artificially altered to create different copies. Different positions relate, for example, to a linear displacement and / or a rotation of the obstacle. By way of example, reference is again made here to a door, the closed position of which is artificially altered to generate different copies.

[0022] It is also conceivable, alternatively or additionally, to artificially change the color of the object to generate a different copy.

[0023] It is preferred that the original recording is generated at a certain distance from the object. This means that the original recording does not show the object from immediately close by. This enables the copy to be generated in an improved manner and leads to a more reliable bypassing of the obstacle.

[0024] It is to be understood that, in addition to the mentioned method, a system that carries out the method also falls within the scope of protection of the present application.

[0025] The system comprises a robot and an IT infrastructure.

[0026] The robot advantageously has an optical recording device for generating the optical recording and a movement device for automatically moving the robot. The optical recording device is preferably also used to carry out the movement routine, that is to say in particular to navigate the robot through the room. The original recording is thus generated by the robot, wherein the optical recording device, which is also used to carry out the movement routine, is preferably used. This leads to a simplified structure of the robot and thus to a low-cost manner of manufacture. The artificial intelligence is implemented in the IT infrastructure. The IT infrastructure thus comprises an artificial intelligence, in particular a neural network. The system is designed here to carry out the method according to the application.

[0027] In particular, the system is designed in such a way that the robot generates the original recording using the recording device and transmits it to the IT infrastructure. The system is also designed in such a way that the IT infrastructure generates the copy and carries out the training process and, if necessary, also the corresponding test process.

[0028] In principle, the IT infrastructure can be completely separate from the robot.

[0029] The following variant is advantageous, wherein the robot has a control device as a component of the IT infrastructure. It is advantageous here that the control device is used in particular to carry out the movement routine. Furthermore, the IT infrastructure has a main structure separate from the robot, which comprises an artificial intelligence, in particular a neural network. The resource-intensive processes of the IT infrastructure are thus carried out outside the robot. The robot can thus be manufactured simply and at low cost. Furthermore, the main structure can be used for a large number of robots, thereby making the overall system design simpler and less costly. The system advantageously comprises a communication device via which the robot, in particular the control device, and the main structure communicate with one another. The communication device is in particular designed for wireless communication between the robot and the main structure.

[0030] Here, the robot generates, in particular with the recording device and the control device, a raw recording. Here, the robot can move away from the obstacle after the collision with the obstacle in order to generate the raw recording at a certain distance from the obstacle. The raw recording is transmitted to the main structure by means of the communication device. If there is a training result, this is preferably transmitted to the robot by means of the communication device, which takes this into account in order to bypass the obstacle in the future.

[0031] In principle, the main structure can be assigned to the user locally. The main structure can thus be, in particular, a local server, in particular a home server.

[0032] The main structure is preferably implemented as a cloud service. This enables the system to be used in a versatile manner and to be realized overall at low cost.

[0033] The movement device serves to move the robot automatically through the room. To this end, the movement device has, for example, an electric motor or the like.

[0034] The robot advantageously has a device for detecting a collision of the robot with an obstacle, which is also referred to below as a detection device. Collision recognition is thus carried out internally in the robot, so that there is no need to use artificial intelligence continuously in the manner described according to the application, i.e. in order to generate copies and to train. Furthermore, the method according to the application is triggered in this way by the robot.

[0035] In principle, the detection device can be designed in any way as long as it is able to recognize a collision of the robot with an obstacle.

[0036] The direction device can have, for example, at least one tactile sensor, which recognizes a collision with an obstacle by means of a tactile contact with the obstacle. Alternatively or additionally, the detection device can monitor the power consumption of the movement device, for example of an electric motor, in order to recognize a collision on the basis of the power consumption. In particular, a sudden and sharp increase in the power consumption is used in this case as an indicator of a collision. Alternatively or additionally, it is conceivable to recognize a collision with an obstacle by means of a combined use of a plurality of sensors. In particular, information about the power consumption of the detection device can be advantageously combined, for example, with data of an inertial sensor system, which advantageously has a gyroscope and / or an acceleration sensor, in order to distinguish a collision from other states of higher power consumption, for example driving on carpets and other soft ground, with improved reliability.

[0037] Depending on the intended use, the robot can of course also have other components. For example, a cleaning robot can have a cleaning device, for example a suction device for suctioning the floor.

[0038] It should be understood that computer programs including instructions which cause a method according to the application to be performed, in particular instructions which cause the system to perform a method according to the application, as well as the described methods and systems, are within the scope of the present application. Likewise, computer readable media storing such computer programs are within the scope of the present application.

[0039] Further important features and advantages of the present application result from the dependent claims, the drawings and the following description with reference to the drawings.

[0040] It should be understood that the features described above and hereinafter can be used not only in the respective combinations indicated, but also in other combinations or alone, without departing from the scope of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0041] In the drawings preferred embodiments of the present application are shown and explained in more detail in the following description, in which identical reference signs refer to identical or similar or functionally identical components. Therein, respectively schematically:

[0042] Figure 1 A highly simplified symbolic representation of a system with a robot and an IT infrastructure is shown, and

[0043] Figure 2 A flow chart of a method for operating the system is shown. DETAILED DESCRIPTION

[0044] According to the method shown in the flow chart exemplary in Figure 2 the system 1 shown in highly simplified form exemplary in Figure 1

[0045] The system 1 comprises a robot 2 and an IT infrastructure 3. In the exemplary embodiment shown, the robot 2 is a domestic robot 4, for example a cleaning robot 5 for cleaning a not shown household. The robot 2 has an optical recording device 6, preferably for moving and navigating the robot 2. With the optical recording device 6, among other things, an optical recording of the surroundings of the robot 2 can be generated. The robot 2 also has a movement device 7 for automatically moving the robot 2. The movement device 7 can for example have a not shown electric motor which drives at least one not shown wheel of the robot 2. In order to supply the robot 2 with energy, the robot also has an energy store 8, in particular a rechargeable battery 9. The robot 2 designed as a cleaning robot 5 also has a cleaning device 10, for example a suction device 11, with which the robot 2 cleans a not shown room, in particular a household. The robot 2 also has a detection device 12 which is designed to recognize a collision of the robot 2 with a not shown obstacle.

[0046] ​The IT infrastructure 3 comprises components arranged in the robot 2 as well as components separate from the robot 2 and in Figure 1 The control device 13 of the robot 2 is a component of the IT infrastructure 3. The control device 13 is in communication connection with the recording device 6 and the movement device 7. Preferably, the control device is also in communication connection with the detection device 12. In the control device 13, routines 20 for moving the robot 2 are stored (cf. Figure 2 ), which are also referred to as movement routines 20 hereinafter. In the movement routines 20, it is mapped, for example, which rooms the robot 2 should or is allowed to move through. The movement routines 20 can also contain obstacles which the robot 2 should bypass, wherein these obstacles are also referred to as known obstacles hereinafter.

[0047] The IT infrastructure 3 also comprises a main structure 14, which in the example shown and preferably is a cloud service 15. The main structure 14 comprises an artificial intelligence 16, in particular a neural network 17. The robot 2, in particular the control device 13, and the main structure 14 communicate with each other preferably wirelessly via a communication device 18, wherein the communication device 18 has a communication unit 19 at the robot 2 as well as on the main structure 14, respectively.

[0048] According to the flowchart shown as an example in Figure 2 , the robot 2 moves through the room using the movement routines 20. In this case, the surroundings are monitored with the recording device 6. When a known obstacle is recognized, it is bypassed, i.e. the robot 2 is prevented from colliding with the known obstacle. The recording device 6 is thus used for navigating the robot 2. In operation, the robot 2, which is designed as a cleaning robot 5, cleans the room, in particular the not shown floor, by means of the cleaning device 10.

[0049] When the robot 2 collides with such an obstacle, a method for bypassing obstacles not considered in the movement routines 20, also referred to as unknown obstacles hereinafter, is triggered. For this reason, in Figure 2The transition to the subsequent measures 21 is shown by a dashed line. Here, in the case of triggering such a measure 21 of the method, a collision of the robot 2 with an obstacle is recognized, wherein the measure 21 is also referred to in the following as a recognition measure 21. The detection device 12 of the robot 2 serves for this purpose. The collision with the obstacle serves as a reason for assuming that it is an unknown obstacle. In a subsequent measure 22, the robot 2 moves away from the obstacle so that the robot 2 is arranged at a certain distance from the obstacle. This measure 22 is therefore also referred to in the following as a distance measure 22. If the robot 2 is at a certain distance from the obstacle, an optical recording of the obstacle is generated in a measure 23 using the recording device 6, wherein the measure 23 is also referred to in the following as a recording measure 23, wherein the recording is also referred to in the following as an original recording. The original recording is then transmitted to the main structure 14 using the communication device 18.

[0050] The method continues in the main structure 14. In a replication measure 24, a plurality of copies 25 of the original recording are artificially generated in the main structure 14. These copies 25 take into account the geometry of the obstacle and differ from one another. The differences of the copies 25 can be achieved by artificially generated different obstacle positions and / or artificially generated different obstacle colors and / or artificially generated different obstacle backgrounds. The copies 25 are divided into two groups, namely training copies 25a and test copies 25b.

[0051] Then, in a training process 26, the artificial intelligence 16 is trained with the training copy 25a. In the training process 26, the artificial intelligence 16, in particular the neural network 17, is trained in such a way that the robot 2 recognizes the obstacle before the collision in the movement routine 20 and bypasses it. The training process 26 is interrupted intermittently in a test process 27 and is thus paused. In the test process 27, the hitherto targeted results of the training process 26, also referred to below as intermediate results, are tested. In this case, a test copy 25b is used in the test process 27. In the test process 26, a test is carried out with at least a part of the test copy 25b: with the intermediate results, with what probability does the robot 2, in particular the movement routine 20, recognize the obstacle before the collision. If this probability is below a predefined value, the method returns to the training process 26 and continues the training process 26. If the probability is above the predefined value, the intermediate results are retained as training results and used in the movement routine 20 of the robot 2. For this purpose, the main structure 14 transmits the training results to the control device 13 by means of the communication device 18 in order to integrate the training results into the movement routine 20. The integration of the training results into the movement routine 20 can take place within the robot 2, in particular by means of the control device 13. Alternatively, the main structure 14 can integrate the training results into the movement routine 20 and transmit the movement routine 20 taking into account the training results to the robot 2, in particular to the control device 13, which then uses the movement routine 20 taking into account the training results.

[0052] During the training process 26 and the test process 27, the robot 2 can be in normal operation. This means, in particular, that the robot 2 can use the existing movement routine 20 during the training process 26 and the test process 27.

[0053] List of reference signs

[0054] 1 system

[0055] 2 robot

[0056] 3 IT infrastructure

[0057] 4 domestic robot

[0058] 5 cleaning robot

[0059] 6 recording device

[0060] 7 movement device

[0061] 8 energy store

[0062] 9 battery

[0063] 10 cleaning device

[0064] 11 suction device

[0065] 12 detection device

[0066] 13 control device

[0067] 14 main structure

[0068] 15 cloud service

[0069] 16 artificial intelligence

[0070] 17 neural network

[0071] 18 communication device

[0072] 19 communication unit

[0073] 20 mobile routine

[0074] 21 identification measure

[0075] 22 spacing measure

[0076] 23 spacing measure

[0077] 24 copying measure

[0078] 25 copy

[0079] 26 training process

[0080] 27 testing process

Claims

1. A computer-implemented method of bypassing an impassable obstacle by a robot (2), wherein, When the robot moves through a room, a collision of the robot (2) with an obstacle is recognized; wherein an optical raw recording of the obstacle is generated; wherein a plurality of artificial copies (25) is generated from the raw recording, wherein the copies each take into account the geometry of the obstacle and differ from one another, so that the number of raw recordings of the respective obstacle required for training artificial intelligence is reduced, wherein in generating the artificial copies the geometry of the obstacle is slightly changed in at least a part of the copies; wherein in a training process (26) at least a part of the copies (25) is used to train an artificial intelligence (16) such that the robot (2) in the future recognizes the obstacle before colliding with it; wherein the training result of the training process (26) is used to bypass the obstacle by the robot (2) in the movement routine of the robot (2), wherein training copies (25a) and test copies (25b) are generated from the raw recording; the artificial intelligence (16) is trained in the training process (16) using the training copies (25a); the training process (26) is interrupted intermittently and the intermediate result of the training process (26) is tested in a test process (27) using at least a part of the test copies (25b) as to what probability the robot (26) recognizes the obstacle before the collision; if the probability is above a predefined value, the intermediate result is used as training result and if the probability is below a predefined value, the training process continues and is then again tested by the test process in intervals.

2. The method of claim 1, wherein, At least a part of the copies (25) is generated such that the surroundings of the obstacle differ from one another in the copies (25).

3. The method according to claim 1 or 2, characterized in that, At least a part of the copies (25) is generated such that the position of the obstacle differs from one another in the copies (25).

4. The method according to claim 1 or 2, characterized in that, The raw recording is generated at a distance from the obstacle.

5. The method of claim 1, wherein, The robot (2) is a domestic robot.

6. The method of claim 1, wherein, The artificial intelligence (16) is a neural network.

7. A system (1) with a robot (2) and an IT infrastructure (3); wherein, The robot (2) has an optical recording device (6) for generating an optical recording and a movement device (7) for moving the robot (2); wherein the IT infrastructure (3) comprises an artificial intelligence (16); wherein the system (1) is designed to carry out the method according to any one of claims 1 to 6.

8. The system of claim 7, wherein, The robot (2) has a control device (13) as a component of the IT infrastructure (3); the IT infrastructure (3) has a main structure (14) separate from the robot (2), which comprises the artificial intelligence (16); the system (1) has a communication device (19) for communication between the control device (13) and the main structure (14).

9. The system of claim 7 or 8, wherein, The robot (2) has a detection device (12) for detecting a collision of the robot (2) with an obstacle.

10. The system of claim 7, wherein, The robot (2) is a domestic robot.

11. The system of claim 7, wherein, The artificial intelligence (16) is a neural network.

12. A computer program product comprising instructions causing the system (1) of any one of claims 7 to 11 to perform the method of any one of claims 1 to 6.

13. A computer readable medium having stored thereon the computer program product of claim 12.

Images