Robotic cleaning device using optical sensor for navigation

Abstract

The present disclosure relates to a method of navigating a robotic cleaning device (200) on a surface to be cleaned, and a robotic cleaning device (200) performing the method. In one aspect, a robotic cleaning device (200) is provided that is configured to navigate on a surface to be cleaned. The robotic cleaning device (200) comprises: a propulsion system (112, 113, 115a, 115b) configured to move the robotic cleaning device (200) over a surface to be cleaned; a camera (123) configured to capture an image of the surroundings of the robotic cleaning device (200); at least one light source (127) configured to illuminate an object in front of the camera (123); an optical ranging sensor (129) arranged towards the surface and configured to measure a position of the robotic cleaning device (200); a heading sensor (122a, 122b, 124, 129) configured to measure a heading of the robotic cleaning device (200); and a controller (116) configured to detect a light section in each captured image resulting from illuminating the object by the at least one light source (127), the light section representing the detected object data, using the measured position and heading of the robotic cleaning device (200) to determine the position of the object data detected in each captured image relative to a reference position, and generating a 3D representation of the illuminated object by aggregating the detected object data of the captured image taking into account the position at which the detected object data of the captured image has been determined, with which the robotic cleaning device (200) navigates on the surface to be cleaned.

Description

Technical Field

[0001] The present disclosure relates to a method of navigating a robotic cleaning device over a surface to be cleaned, and a robotic cleaning device performing the method. Background Art

[0002] In many areas of technology it is desirable to use robots with autonomous behavior such that they can move freely around in space without colliding with possible obstacles.

[0003] Robotic vacuum cleaners are known in the art, which are equipped with drive means in the form of motors for moving the cleaner over the surface to be cleaned. The robotic vacuum cleaners are further equipped with intelligence in the form of microprocessor(s) and navigation means for causing autonomous behavior, enabling the robotic vacuum cleaner to move around freely and clean surfaces in the form of, for example, floors. These prior art robotic vacuum cleaners therefore have the ability to more or less autonomously vacuum clean rooms in which objects, such as tables and chairs, and other obstacles, such as walls and stairs, are located.

[0004] To navigate on the surface to be cleaned, prior art robotic vacuum cleaners typically use a camera for capturing images of the surrounding environment, from which a 2D or 3D representation of the surrounding environment is generated. Further, such robotic vacuum cleaners typically use wheel encoders as well as gyroscope data to determine the position and heading of the robotic vacuum cleaner relative to a reference position. The wheel encoders determine the distance traveled by the robot by counting wheel rotations, and the robot uses the determined distance to navigate on the surface. This is known as dead reckoning.

[0005] The problem with these prior art robot vacuum cleaners is that they are sensitive to wheel slip. When travelling over surfaces such as glossy parquet or rugs, the wheels of the robot will occasionally slip and, as a result, the wheel encoders will not be able to correctly determine the distance travelled, resulting in erroneous navigation. Summary of the invention

[0006] An object is to solve or at least alleviate this problem in the prior art and therefore to provide an improved method of navigating a robotic cleaning device over a surface to be cleaned.

[0007] In a first aspect, the object is achieved by a robotic cleaning device configured to navigate on a surface to be cleaned, the robotic cleaning device comprising: a propulsion system configured to move the robotic cleaning device on the surface to be cleaned; a camera configured to capture images of the surrounding environment of the robotic cleaning device; at least one light source configured to illuminate an object in front of the camera; and an optical odometry sensor arranged to face the surface and configured to measure the position of the robotic cleaning device. The robotic cleaning device further comprises: a heading sensor configured to measure the heading of the robotic cleaning device; and a controller configured to detect a bright segment in each captured image caused by the at least one light source illuminating the object (the bright segment represents detected object data), determine the position of the detected object data in each captured image relative to a reference position using the measured position and heading of the robotic cleaning device, generate a 3D representation of the illuminated object by aggregating the detected object data of the captured images taking into account the determined position of the detected object data of the captured images, and the robotic cleaning device navigates on the surface to be cleaned using the generated 3D representation.

[0008] In a second aspect, the object is achieved by a method for navigating a robotic cleaning device on a surface to be cleaned. The method comprises: illuminating an object in front of a camera of the robotic cleaning device; capturing images of the surroundings of the robotic cleaning device; measuring the position of the robotic cleaning device using an optical odometry sensor arranged to face the surface and measuring the heading of the robotic cleaning device using a heading sensor as the robotic cleaning device moves over the surface; detecting a bright segment in each captured image caused by the illumination of the object (the bright segment representing detected object data); determining the position of the detected object data in each captured image relative to a reference position using the measured position and heading of the robotic cleaning device; and generating a 3D representation of the illuminated object by aggregating the detected object data of the captured images taking into account the determined positions of the detected object data of the captured images, the robotic cleaning device navigating on the surface to be cleaned using the generated 3D representation.

[0009] Advantageously, by using an optical sensor to determine the position of the robotic cleaning device, the robotic cleaning device will not be affected by wheel slip when generating a 3D representation that enables the robotic cleaning device to navigate the surface to be cleaned.

[0010] Thus, even if the robotic device is traveling over a surface that would cause the robotic device wheels to slip, the optical sensors facing the surface will correctly measure the position (and possibly heading) of the robotic device relative to a selected reference point.

[0011] In an embodiment, the at least one light source comprises first and second line lasers configured to illuminate an object in front of the camera.

[0012] In an embodiment, the first and second line lasers are vertically oriented line lasers.

[0013] In an embodiment, the at least one light source further comprises a horizontally oriented line laser.

[0014] In an embodiment, the first and second line lasers are arranged symmetrically on opposite sides of the camera.

[0015] In an embodiment, the robotic cleaning device further comprises an inertial measurement unit configured to measure a heading of the robotic cleaning device.

[0016] In an embodiment, the robotic cleaning device further comprises an odometry encoder disposed on each drive wheel of the propulsion system to measure the position and heading of the robotic cleaning device.

[0017] In an embodiment, the heading sensor is one of an optical odometry sensor, an inertial measurement unit, an odometry encoder, or a combination thereof.

[0018] In an embodiment, the optical sensor is arranged in a recess on the underside of the body of the robotic cleaning device to protect the optical sensor from being hit by any object.

[0019] In an embodiment, the optical sensor is disposed behind an opening in the body of the robotic cleaning device such that debris and dust are removed from the surface toward which the optical sensor is directed.

[0020] In a third aspect, there is provided a computer program comprising computer executable instructions for causing the robotic cleaning device of the first aspect to perform the steps of the method of the second aspect when the computer executable instructions are executed on a controller comprised in the robotic cleaning device.

[0021] In a fourth aspect, there is provided a computer program product comprising a computer readable medium having embodied thereon the computer program according to the third aspect.

[0022] In general, unless otherwise explicitly defined herein, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field. Unless otherwise explicitly stated, all references to "a / an / the element, device, component, device, step, etc." are to be interpreted openly as referring to at least one example of the element, device, component, device, step, etc. Unless explicitly stated, the steps of any method disclosed herein do not have to be performed exactly in the order disclosed. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Various aspects and embodiments will now be described by way of example with reference to the accompanying drawings, in which:

[0024] Figure 1 shows a bottom view of a prior art robotic cleaning device in which an embodiment may be implemented;

[0025] Figure 2 Shows Figure 1 A front view of a robotic cleaning device;

[0026] Figure 3a to Figure 3c Shown Figure 1 and Figure 2 A robotic cleaning device moves around over the surface to be cleaned in order to generate a 3D representation of the surrounding environment;

[0027] Figure 4 Demonstrated incorrect robot navigation caused by wheel slip;

[0028] Figure 5 shows a bottom view of a robotic cleaning device according to an embodiment; and

[0029] Figure 6 A flow chart illustrating a method of navigating a robotic cleaning device over a surface to be cleaned according to an embodiment is shown. DETAILED DESCRIPTION

[0030] Various aspects of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings, in which certain embodiments are shown.

[0031] However, these aspects can be implemented in many different forms and should not be interpreted as limiting; rather, these embodiments are provided by way of example so that this disclosure will become comprehensive and complete and will fully convey the scope of all aspects to those skilled in the art. Throughout the specification, the same reference numerals refer to the same elements.

[0032] Embodiments relate to a robotic cleaning device, or in other words, to an autonomous self-propelled machine for cleaning surfaces, such as a robotic vacuum cleaner, a robotic sweeper or a robotic floor washer. The robotic cleaning device according to the invention can be operated with mains power and have an electric cord, can be operated with batteries or use any other kind of suitable energy source, such as solar energy.

[0033] While it is contemplated that the present invention may be implemented by any suitable robotic cleaning device equipped with sufficient processing intelligence, Figure 1 A bottom view of a prior art robotic cleaning device 100 is shown, ie, the underside of the robotic cleaning device is shown, in which embodiments may be implemented. The arrow indicates the forward direction of the robotic cleaning device 100, which is shown in the form of a robotic vacuum cleaner.

[0034] The robot cleaning device 100 includes a main body 111, which accommodates components such as a propulsion system, which includes a drive device in the form of two electric wheel motors 115a, 115b that can move drive wheels 112, 113, so that the cleaning device can move on the surface to be cleaned. Each wheel motor 115a, 115b can control the corresponding drive wheels 112, 113 to rotate independently of each other so that the robot cleaning device 100 moves on the surface to be cleaned. Many different drive wheel arrangements and various wheel motor arrangements can be envisioned. It should be noted that the robot cleaning device can have any suitable shape, such as a device with a more traditional circular shape body or a triangular shape body. As an alternative, a tracking propulsion system or even a hovercraft propulsion system can be used. The propulsion system can be further arranged to enable the robot cleaning device 100 to perform any one or more of yaw translation or pitch translation or roll movement.

[0035] The controller 116 (such as a microprocessor) receives the obstacle detection device ( Figure 1 The microprocessor 116 controls the wheel motors 115a, 115b to rotate the drive wheels 112, 113 as required based on information received from the obstacle detection device (not shown in the figure), which is used to detect obstacles in the form of walls, floor lamps, table legs, around which the robotic cleaning device must navigate. The obstacle detection device can be implemented in the form of a 3D sensor system that registers its surroundings (implemented by means of, for example, a 3D camera, a camera combined with a laser, a laser scanner, a laser projector, or a light-emitting diode (LED) illuminator, etc.), detects obstacles and communicates information about any detected obstacles to the microprocessor 116. The microprocessor 116 communicates with the wheel motors 115a, 115b to control the movement of the wheels 112, 113 based on the information provided by the obstacle detection device, so that the robotic cleaning device 100 can move on the surface to be cleaned as required.

[0036] Further, the body 111 may optionally be provided with a cleaning member 117 for removing debris and dust from the surface to be cleaned, in the form of a rotatable brush roller arranged in an opening 118 at the bottom of the robotic cleaner 100. Thus, the rotatable brush roller 117 is arranged in the opening 118 along a horizontal axis to enhance the dust and debris collection characteristics of the robotic cleaning device 100. To rotate the brush roller 117, a brush roller motor 119 is operably coupled to the brush roller to control its rotation according to instructions received from the controller 116.

[0037] Moreover, the body 111 of the robot cleaner 100 includes a suction fan 120 that generates an airflow to transport debris to a dust bag or cyclone arrangement (not shown) housed in the body via an opening 118 in the bottom side of the body 111. The suction fan 120 is driven by a fan motor 121 communicatively connected to the controller 116, which receives instructions for controlling the suction fan 120 from the controller. It should be noted that a robot cleaning device having either a rotatable brush roller 117 and a suction fan 20 to transport debris to a dust bag can be envisioned. However, a combination of the two will enhance the debris removal capability of the robot cleaning device 100.

[0038] The main body 111 of the robot cleaning device 100 may be further equipped with an inertial measurement unit (IMU) 124, such as a gyroscope and / or an accelerometer and / or a magnetometer or any other suitable device, for measuring the displacement of the robot cleaning device 100 relative to a reference position in the form of, for example, orientation, rotation speed, gravity, etc. The three-axis gyroscope can measure the rotation speed of the roll, pitch, and yaw movement of the robot cleaning device 100. The three-axis accelerometer can measure the acceleration in all directions, which is mainly used to determine whether the robot cleaning device is blocked or lifted or stuck (that is, even if the wheel is rotating but not moving). The robot cleaning device 100 further includes encoders 122a, 122b on each drive wheel 112, 113, which generate pulses when the wheel rotates. The encoder can be, for example, magnetic or optical. By counting pulses at the controller 116, the speed of each wheel 112, 113 can be determined. From the wheel speed readings, the controller 116 can perform so-called dead reckoning to determine the position and heading of the cleaning device 100. This can be further improved by also taking into account gyroscope information in addition to the wheel speed readings.

[0039] The main body 111 may further be provided with a rotating side brush 114 adjacent to the opening 118, the rotation of which may be controlled by a drive motor 115a, a drive motor 115b, a brush roller motor 119, or alternatively a separate side brush motor (not shown). Advantageously, the rotating side brush 114 sweeps up debris and dust from the surface to be cleaned so that the debris ends up under the main body 111 at the opening 118 and can thus be transported to a dust collection chamber of the robotic cleaning device. Further advantageously, the reach of the robotic cleaning device 100 will be improved, and areas such as corners and where the floor meets the wall will be cleaned more efficiently. Figure 1 As shown, the rotating side brush 114 rotates in one direction so that it sweeps up debris toward the opening 118 so that the suction fan 120 can transport the debris to the dust collection chamber. The robotic cleaning device 100 may include two rotating side brushes that are laterally disposed on each side of the opening 118 and adjacent to the opening.

[0040] Further references Figure 1 , the controller / processing unit 116 implemented in the form of one or more microprocessors is arranged to execute a computer program 125 downloaded to a suitable storage medium 126 associated with the microprocessor (e.g., a random access memory (RAM), flash memory, or a hard drive). The controller 116 is arranged to implement a method according to an embodiment of the present invention when a suitable computer program 125 including computer executable instructions is downloaded to the storage medium 126 and executed by the controller 116. The storage medium 126 may also be a computer program product including the computer program 125. Alternatively, the computer program 125 may be transferred to the storage medium 126 via a suitable computer program product (e.g., a digital versatile disk (DVD), a compact disk (CD), or a memory stick). As a further alternative, the computer program 125 may be downloaded to the storage medium 126 via a wired or wireless network. The controller 116 may alternatively be implemented in the form of a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a complex programmable logic device (CPLD), etc.

[0041] Figure 2 Shows Figure 1 1, showing the aforementioned obstacle detection device in the form of a 3D sensor system, which includes at least a camera 123 and a first line laser 127 and a second line laser 128, which can be horizontally or vertically oriented lasers. Figure 1The controller 116, the main body 111, the drive wheel 112, the drive wheel 113 and the rotatable brush roller 117 discussed. The controller 116 is operably connected to a camera 123 for recording images of the vicinity of the robot cleaning device 100. The first line laser 127 and the second line laser 128 may preferably be vertical line lasers and are arranged on the side of the camera 123 and are configured to illuminate a height and width greater than the height and width of the robot cleaning device 100. Further, the field of view angle of the camera 123 is preferably greater than the space illuminated by the first line laser 127 and the second line laser 128. The controller 116 controls the camera 123 to capture and record multiple images per second. Data is extracted from these images by the controller 116, and the data is typically stored in the memory 126 together with the computer program 125.

[0042] The first line laser 127 and the second line laser 128 are typically arranged on respective sides of the camera 123 along an axis perpendicular to the optical axis of the camera. Further, the line lasers 127, 128 are oriented so that their respective laser beams intersect within the field of view of the camera 123. Typically, the intersection coincides with the optical axis of the camera 123.

[0043] The first line laser 127 and the second line laser 128 are configured to scan the vicinity of the robotic cleaning device 100, preferably in a vertical orientation, normally in the direction of movement of the robotic cleaning device 100. The first line laser 127 and the second line laser 128 are configured to emit laser beams that illuminate, for example, furniture, walls, and other objects in the room to be cleaned. When the robotic cleaning device 100 moves over the surface to be cleaned, the controller 116 controls the camera 123 to capture and record images, and by extracting features from these images and by measuring the distance covered by the robotic cleaning device 100, the controller 116 generates a representation or layout of the surrounding environment in which the robotic cleaning device 100 operates from these images. Therefore, the controller 116 obtains the position data of the robotic cleaning device 100 relative to the surface to be cleaned from the wheel encoders 122a, 122b and the IMU 124, generates a 3D representation of the surrounding environment from the obtained position data (which is associated with the extracted features), and controls the drive motors 115a, 115b to move the robotic cleaning device on the surface to be cleaned according to the generated 3D representation and navigation information provided to the robotic cleaning device 100, so that the surface to be cleaned can be navigated in consideration of the generated 3D representation. Because the obtained position data will be used as the basis for navigation of the robotic cleaning device, correct positioning is important; otherwise, the robotic device will navigate based on a misleading "map" of its surrounding environment.

[0044] Thus, the 3D representation generated from the images recorded by the 3D sensor system helps detect obstacles in the form of walls, floor lamps, table legs around which the robotic cleaning device must navigate, as well as obstacles in the form of rugs, carpets, doorsteps, etc. that the robotic cleaning device 100 must traverse. Thus, the robotic cleaning device 100 is configured to learn its environment or surroundings by moving around over the surface to be cleaned.

[0045] Thus, a 3D sensor system comprising a camera 123 and a first vertical line laser 127 and a second vertical line laser 128 is arranged to record images of the surroundings of the robot cleaning, from which objects / obstacles can be detected. The controller 116 is able to position the detected obstacles relative to a reference position on the surface to be cleaned to generate a 3D representation. Starting from positioning, the controller 116 controls the robot cleaning device 100 to move on the surface to be cleaned by controlling the wheels 112, 113 via wheel drive motors 115a, 115b, thereby enabling navigation.

[0046] As will be appreciated, a single line laser may be used. However, by using dual line lasers 127, 128 a greater amount of data is obtained from which a 3D representation of the surrounding environment is generated (typically twice as much data is obtained using dual lasers).

[0047] The obtained position of any detected object relative to the reference position helps control the movement of the robotic cleaning device 100 so that the cleaning device can be navigated to move very close to the object and move closely around the object to remove debris from the surface on which the object is located. Therefore, the obtained position data is used to move flush with the object (e.g., a chair, table, sofa, thick small rug or wall). Typically, the controller 116 continuously generates and transmits control signals to the drive wheels 112, 113 via the drive motors 115a, 115b so that the robotic cleaning device 100 is navigated close to the object.

[0048] Figure 3a Demonstrated object detection on a surface over which a robotic cleaning device moves.

[0049] Thus, the robotic device 100 records images while traveling over the surface to be cleaned, from which a 3D representation is generated for navigation of the device.

[0050] In this particular exemplary embodiment, the robotic device 100 uses two vertical line lasers 127, 128 to illuminate the surface on which the robotic device moves. Figure 3a As can be seen in FIG. 1 , each of the line lasers 127 , 128 projects a laser beam onto the floor and the first wall of the room to be cleaned, while the robotic device 100 uses its camera 123 to capture images of the illuminated surfaces.

[0051] As can be seen, the laser beam will fall on the floor and the wall, resulting in two parallel lines in the captured image. Based on the captured image, the robotic device 100 detects that the laser beam illuminates an obstacle, in this case a wall. The robotic device 100 is able to determine the distance to the wall based on the captured image. Thus, the robotic cleaning device 100 determines the position of the obstacle relative to the reference position of the navigation coordinate system.

[0052] By moving over the surface and using the data recorded by the encoders 122a, 122b to determine the distance traveled and using the data from the IMU 124 to determine the heading, the robotic cleaning device 100 calculates the position of the detected object data (i.e., the data represented by the bright segments corresponding to the line laser shining on the wall in each captured image) relative to the reference position of the navigation coordinate system. As will be appreciated, this reference position may constitute any appropriately selected position in the room. The navigation coordinate system may be fixed relative to the robotic cleaning device 100 itself or relative to the surrounding environment using, for example, a reference position somewhere on the floor.

[0053] Figure 3b The robotic cleaning device 100 is shown rotating slightly to the right and capturing a second image (the laser beam moves corresponding to this rotation), while Figure 3c The robotic cleaning device 100 is shown rotating further to the right and capturing a third image.

[0054] By capturing multiple images and aggregating the detected object data for each image, the robotic device 100 is able to generate a 3D representation that reproduces a particular obstacle with high reliability.

[0055] As described, for Figure 3a to Figure 3c As shown, for each image captured, the robotic device 100 must track its movement and heading relative to the navigation coordinate system reference position because a 3D representation of the surrounding environment will be generated by combining the detected object data of each captured image, and thus correctly position the detected objects in the navigation coordinate system.

[0056] Therefore, it is important to correctly compute the distances between the detected object data in each image using the dead reckoning approach described previously.

[0057] However, reference Figure 4 As shown, the problem with using wheel encoders 122a, 122b to determine the movement of the robotic cleaning device 100 is that if wheel slip occurs (for example, if the robotic device 100 is traveling over a high gloss floor or a small rug), the calculated movement will be incorrect (the estimated travel distance will be greater than the actual distance).

[0058] Therefore, because the aggregation of the detected object data taking into account the movement and heading of the robotic device 100 relative to the reference position is performed using erroneous movement data recorded by the wheel encoders 122a, 122b, an incorrect 3D representation will eventually be generated.

[0059] exist Figure 4 , the solid lines show the actual room boundaries, while the dashed lines show the incorrect perception of the room by the robotic device 100 when performing dead reckoning based on distances incorrectly calculated due to wheel slip.

[0060] Now, even if the 3D representation is corrected, wheel slippage during navigation using the corrected 3D representation will result in incorrect distance calculations and incorrect navigation of the robot relative to the 3D representation. Further, the 3D representation is repeatedly updated, for example, because furniture such as a chair can be added to a room, the chair should also be added to the 3D representation by capturing additional images and detecting new obstacles from these images to be added to the 3D representation. Therefore, wheel slippage will result in incorrect updates of the 3D representation.

[0061] A commonly used approach is to use LiDAR (“Light Detection and Ranging”), also known as 2D scanning, where distance measurements are collected 360° around the robot many times per second, thereby obtaining reliable distance data for robust navigation as well as overlapping images to form a 3D representation.

[0062] The disadvantage of using LiDAR is that only obstacles at the height of the LiDAR sensor turret can be registered. The LiDAR collects a rotating disc of distance data from the sensor, which usually protrudes about 10 mm from the upper side of the robot. Therefore, data above and (more importantly) below the LiDAR disc cannot be registered. In order to detect obstacles, the robot must be equipped with additional sensors to cover the gap between the floor and the LiDAR sensor.

[0063] Alternatively, a known 3D sensor using time-of-flight measurements of a fully illuminated image may be used. Using such a time-of-flight 3D sensor, the distance in the captured image will be determined for each pixel, and thus the distance of the detected object data can be determined even if wheel slip occurs. However, in addition to being expensive, a larger amount of data must be processed, which increases the processing burden of the processing unit 116. Typically, in the case of using a time-of-flight 3D sensor, the robotic device 100 will have to be equipped with a more powerful processor.

[0064] refer to Figure 5In order to solve the problem that wheel slip causes the wheel encoders 122a, 122b to incorrectly register the movement of the robotic cleaning device 100, a robotic cleaning device 200 is provided. In an embodiment, the robotic cleaning device is equipped with an optical sensor 129 arranged on the underside of the robotic device 200 and facing the floor to measure the movement and heading of the robotic device 200.

[0065] In an embodiment, the optical sensor 129 is disposed in a recess on the underside of the body 111 of the robotic cleaning device 200 to protect the optical sensor 129 from being hit by any object.

[0066] In an embodiment, the optical sensor 129 is disposed behind the opening 118 of the body 111 of the robotic cleaning device 200 such that debris and dust are removed from the surface toward which the sensor 129 is directed.

[0067] As will be appreciated, such a robotic device 200 does not require Figure 1 The wheel encoders 122a, 122b of the robotic device 100 may be used, but the IMU 124 may be used optionally for more reliable heading measurement. However, in addition to the optical sensor 129, the robotic cleaning device 200 may include the wheel encoders 122a, 122b. Further, the robotic cleaning device 200 of this embodiment uses Figure 2 The camera and line laser setup shown in .

[0068] Figure 5 The robotic cleaning device 200 is shown equipped with such an optical odometry sensor 129 directed toward the floor. The optical sensor 129 is typically in communication with the processing unit 116 so that the processing unit 116 receives movement and heading data recorded by the optical sensor 129 in order to determine the position of the robotic device 200 relative to the reference position of the navigation coordinate system (and therefore the position of any detected object data extracted from the captured images used to generate the 3D representation).

[0069] In an example, the optical sensor 129 includes a camera that captures images of the surface on which the robotic device 200 travels and determines the movement and heading of the robotic device 200. The functionality of the optical sensor 129 is similar to that of a computer mouse; that is, it is able to detect features of a surface in a captured image and determine how those features have moved relative to previously captured images.

[0070] Thus, in the event of wheel slip, the optical sensor 129 will record the actual movement of the robotic device 200 on the floor, rather than the actual movement of the robotic device 200 on the floor. Figure 1 The situation of the robotic device 100 records the movement of the wheels.

[0071] Figure 6A flow chart showing a method of navigating a robotic cleaning device 200 on a surface to be cleaned according to an embodiment is shown. Figure 3a to Figure 3c , wherein the robotic cleaning device 200 navigates over the surface to be cleaned.

[0072] Thus, the robotic device 200 records images while traveling over the surface to be cleaned, from which a 3D representation is generated for navigation of the device 200 .

[0073] In this particular exemplary embodiment, in step S101, the robotic device 100 uses two vertical line lasers 127, 128 to illuminate the surface on which the robotic device moves. Figure 3a As can be seen in FIG. 1 , in step S102 , each of the line lasers 127 , 128 projects a laser beam onto the floor and the first wall of the room to be cleaned, while the robotic device 100 uses its camera 123 to capture images of the illuminated surfaces.

[0074] In an embodiment, the robotic device 200 is equipped with a horizontal line laser (not shown) in addition to utilizing the two vertical line lasers 127, 128. Advantageously, such a third horizontal line laser will be used as a lidar.

[0075] In step S103, as the robotic device 200 moves over the surface to be cleaned, the optical sensor 129 facing the surface measures the movement, and possibly also the heading, of the robotic device 200. Thus, the heading of the robotic cleaning device 200 may be measured using the optical sensor 129, the IMU 124, or even the wheel encoders 122a, 122b (or a combination thereof). In the following example, the optical sensor 129 is used to measure both position and heading.

[0076] As previously referenced Figure 3a As described, the laser beam will fall on the floor and the wall, so that two bright segments appear in the captured image in the form of parallel lines, and in step S104, the controller 116 detects these lines as object data.

[0077] Further, in step S105 , based on the measured movement and heading of the robotic cleaning device 200 , the position of the object data detected in each captured image relative to a reference position of the surrounding environment is determined.

[0078] Thus, using the measured movement and heading data of the optical sensor 129, the controller 116 is able to calculate the position of the detected object data for each captured image relative to a reference point in the room.

[0079] exist Figure 3a In the embodiment, the robotic device 200 captures a first image when it is in a first position and has a first heading relative to a reference point, and in the embodiment of the present invention, the robotic device 200 captures a first image when it is in a first position and has a first heading relative to a reference point, and Figure 3b In the embodiment of the present invention, the second image is captured when the robotic device 200 is in a second position and has a second heading relative to the reference point.

[0080] In other words, the controller 116 determines the position of the detected object data for each captured image based on the position and heading of the robotic device 200 measured by the optical sensor 129, the IMU 124, or the wheel encoders 122a, 122b.

[0081] Thus, the detected object data of the second image is aggregated to the detected object data of the first image by taking into account the determined positions of the two sets of detected object data, thereby allowing the controller 116 to generate a 3D representation of the detected object (in this case the wall) and ultimately its surroundings.

[0082] Similarly, in Figure 3c In the embodiment of the present invention, the robotic device 200 captures a third image when it is in a third position and has a third heading relative to the reference point. The detected object data of the third image are then aggregated to the previously detected object data taking into account the determined position of the detected object data of the third image, thereby advantageously further constructing a 3D representation, in view of which the robotic device 200 performs its navigation on the surface to be cleaned.

[0083] Advantageously, by using the optical sensor 129, the generation of a 3D representation that enables the robotic cleaning device 200 to navigate over the surface to be cleaned will not be affected by wheel slip.

[0084] Thus, even if the robotic device 200 is traveling over a surface that would cause the wheels 112, 113 to slip, the optical sensor 129 facing the surface will correctly measure the position and heading of the robotic device 200 relative to the selected reference point.

[0085] In another embodiment, the robotic device 200 uses the IMU 124 or wheel encoders 122a, 122b to measure the current heading, and the heading data measured by the IMU 124 or wheel encoders 122a, 122b can be used to determine the position of any detected object data relative to a selected reference position, rather than using the heading data measured by the optical sensor 129.

[0086] Aspects of the disclosure have mainly been described above with reference to a few embodiments and examples thereof. However, as readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.

[0087] Therefore, although various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration, not limitation, with the true scope and spirit being indicated by the appended claims.

Claims

1. A robotic cleaning device (200), the robotic cleaning device being configured to navigate on a surface to be cleaned, the robotic cleaning device comprising: a propulsion system (112, 113, 115a, 115b) configured to move the robotic cleaning device (200) over the surface to be cleaned; a camera (123) configured to capture images of the environment surrounding the robotic cleaning device (200); at least one light source (127) configured to illuminate an object in front of the camera (123); an optical odometry sensor (129) disposed toward the surface and configured to measure the position of the robotic cleaning device (200); a heading sensor (122a, 122b, 124, 129) configured to measure the heading of the robotic cleaning device (200); and A controller (116) configured to: detecting a bright segment in each captured image caused by the at least one light source (127) illuminating an object, the bright segment representing detected object data; Using the measured position and heading of the robotic cleaning device (200) to determine the position of the object data detected in each captured image relative to a reference position; The detected object data of the captured images are aggregated to generate a 3D representation of the illuminated object by taking into account the positions determined by the detected object data of the captured images, and the robotic cleaning device (200) navigates on the surface to be cleaned using the generated 3D representation.

2. The robotic cleaning device (200) of claim 1, wherein the at least one light source comprises: A first line laser (127) and a second line laser (128) are configured to illuminate an object in front of the camera (123).

3. The robotic cleaning device (200) of claim 2, the first line laser (127) and the second line laser (128) being vertically oriented line lasers.

4. The robotic cleaning device (200) of claim 2, the at least one light source (127) further comprising a horizontally oriented line laser.

5. The robotic cleaning device (200) according to any one of claims 2 to 4, the first line laser (127) and the second line laser (128) being symmetrically arranged on opposite sides of the camera (123).

6. The robotic cleaning device (200) of any one of the preceding claims, further comprising: An inertial measurement unit (124) configured to measure the heading of the robotic cleaning device (200).

7. The robotic cleaning device (200) of any one of the preceding claims, further comprising: An odometry encoder (122a, 122b) is disposed on each drive wheel (112, 113) of the propulsion system to measure the position and heading of the robotic cleaning device (200).

8. The robotic cleaning device (200) of claims 6 and 7, wherein the heading sensor (122a, 122b, 124, 129) is one of the optical odometry sensor (129), the inertial measurement unit (124), the odometry encoder (122a, 122b), or a combination thereof.

9. The robotic cleaning device (200) according to any of the preceding claims, the optical odometry sensor (129) being arranged in a recess on the underside of the body (111) of the robotic cleaning device (200).

10. The robotic cleaning device (200) of any one of the preceding claims, the optical odometry sensor (129) being arranged behind an opening (118) of a body (111) of the robotic cleaning device (200) through which dust and debris are collected.

11. A method of navigating a robotic cleaning device (200) on a surface to be cleaned, the method comprising: illuminating (S101) an object in front of a camera (123) of the robotic cleaning device (200); capturing (S102) an image of the surrounding environment of the robotic cleaning device (200); As the robotic cleaning device (200) moves over the surface, measuring (S103) the position of the robotic cleaning device (200) using an optical odometry sensor (129) arranged toward the surface, and measuring the heading of the robotic cleaning device (200) using a heading sensor (122a, 122b, 124, 129); detecting ( S104 ) a bright segment in each captured image caused by illumination of the object, the bright segment representing detected object data; Using the measured position and heading of the robotic cleaning device (200), determining (S105) the position of the object data detected in each captured image relative to a reference position; as well as The detected object data of the captured images are aggregated to generate (S106) a 3D representation of the illuminated object by taking into account the positions determined by the detected object data of the captured images, and the robotic cleaning device (100) navigates on the surface to be cleaned using the generated 3D representation.

12. The method of claim 11, the robotic cleaning device (200) comprising a first line laser (127) and a second line laser (128), the first line laser and the second line laser configured to illuminate an object in front of the camera (123).

13. The method of claim 12, the first line laser (127) and the second line laser (128) being vertically oriented line lasers.

14. The method of claim 12, the at least one light source (127) further comprising a horizontally oriented line laser.

15. The method of any one of claims 12 to 14, the first line laser (127) and the second line laser (128) being symmetrically arranged on opposite sides of the camera (123).

16. The method of any one of claims 11 to 15, the robotic cleaning device (200) comprising an inertial measurement unit (124) configured to measure a heading of the robotic cleaning device (200).

17. The method according to any one of claims 11 to 16, wherein the robotic cleaning device (200) comprises an odometry encoder (122a, 122b) arranged on each drive wheel (112, 113) of the propulsion system to measure the position and heading of the robotic cleaning device (200).

18. The method of claims 16 and 17, wherein the heading sensor (122a, 122b, 124, 129) is one of the optical odometry sensor (129), the inertial measurement unit (124), the odometry encoder (122a, 122b) or a combination thereof.

19. The method of any one of claims 11 to 18, the optical odometry sensor (129) being arranged in a recess on the underside of the body (111) of the robotic cleaning device (200).

20. The method of any one of claims 11 to 19, wherein the optical odometry sensor (129) is arranged behind an opening (118) of the body (111) of the robotic cleaning device (200) through which dust and debris are collected.

21. A computer program (125) comprising computer executable instructions to cause the robotic cleaning device (200) to perform the steps of any one of claims 11 to 20 when the computer executable instructions are executed on a controller (116) comprised in the robotic cleaning device (200).

22. A computer program product comprising a computer readable medium (126) on which the computer program (125) as claimed in claim 21 is embodied.

Images