Mapping negative space for autonomous mobile robots

Abstract

Non-temporary machine-readable medium, which, when executed, includes instructions causing a processing circuit to perform an action, the action of receiving sensor data from a mobile cleaning robot, based on interaction between the mobile cleaning robot and the environment. The instructions cause the processing circuit to perform an action to generate, using the sensor data, a boundary of space passable by the mobile cleaning robot within the environment, the boundary of which at least partially defines impassable space within the environment, and the impassable space includes an area beyond the boundary. The instructions cause the processing circuit to perform an action to generate a modified boundary of the environment using the impassable space and sensor data.

Description

Technical Field

[0001] 〔Priority Application〕 This application is a continuation of U.S. Patent Application No. 18 / 118,825, filed on March 8, 2023, the entire content of which is incorporated herein by reference, and claims the priority thereof.

Background Art

[0002] An autonomous mobile cleaning robot can travel on the floor and perform various operations in the environment, such as vacuuming one or more rooms in the environment. The cleaning robot can include a controller configured to autonomously navigate the robot throughout the environment so that it can suck up debris when the robot moves. When the autonomous mobile robot travels on the floor, the robot can generate and record information about the environment and the robot, such as generating a map of the environment for use in the cleaning operation.

Summary of the Invention

Problems to be Solved by the Invention

[0003] Mobile cleaning robots can be used by users, such as homeowners, to perform ad-hoc or scheduled cleaning missions. During a mission, the robot can autonomously navigate the environment and perform cleaning actions such as vacuuming or mopping (or both). While navigating and cleaning, the robot can use its cameras to detect objects in the environment for purposes such as distance measurement, avoidance, or scene understanding. This detection can help the robot perform better cleaning actions, create and use a map of the environment, and avoid sucking up non-debris items. Other sensors on the robot, including wheel encoders, light sensors, or positioning sensors, can also be used to create and update the map. However, when items and rooms are mapped, relatively large items such as beds with impassable spaces underneath may appear as walls and affect the shape of the room as it is displayed. In such cases, the visual representation of the map may be unrecognizable (or difficult to recognize) for the user. [Means for solving the problem]

[0004] This disclosure describes examples of devices, systems, and methods that can help address this problem, such as by using additional data collected by a robot to modify the boundaries of an environment. The modified boundaries may be presented to the user in a modified map in a format that more accurately represents the environment, helping the user to recognize the environment more easily and enabling a quicker and easier setup of the environment, such as naming rooms and spaces. The more accurate map may be used by a robot to clean a space more effectively or efficiently. Furthermore, the data may be used to create a three-dimensional map of the environment to generate an even more realistic representation of the environment.

[0005] Furthermore, some current mapping techniques emphasize the boundaries of traversable spaces, which can change from mission to mission and over time. By using negative space, boundaries can be determined and applied to the map, and these boundaries are more likely to represent the actual boundaries of the environment (e.g., walls) and therefore less likely to move. This can help reduce confusion from users encountering changes within the map.

[0006] For example, a non-temporary machine-readable medium that, when executed, includes instructions causing a processing circuit to perform an action, the action of receiving sensor data from a mobile cleaning robot based on interaction between the mobile cleaning robot and the environment. The instructions cause the processing circuit to perform an action to use the sensor data to generate a boundary of space that the mobile cleaning robot can pass through in the environment, the boundary of which at least partially defines impassable space in the environment, and the impassable space includes an area that extends beyond the boundary. The instructions cause the processing circuit to perform an action to use the impassable space and sensor data to generate a modified boundary of the environment.

[0007] The above description is intended to provide an overview of the subject matter of this patent application. It is not intended to provide an exclusive or exhaustive description of the invention. The following description is included to provide further information about this patent application.

[0008] In drawings, which are not necessarily drawn to a consistent scale, similar numbers may represent similar components in different drawings. Similar numbers with different subscripts may represent different instances of similar components. The drawings schematically illustrate, but are not limiting, the various embodiments described herein. [Brief explanation of the drawing]

[0009] [Figure 1] This is a plan view of a mobile cleaning robot in an environment. [Figure 2A] This is a bottom view of the mobile cleaning robot. [Figure 2B] This is an isometric view of a mobile cleaning robot. [Figure 3] This is a cross-sectional view of the mobile cleaning robot from indicator 3-3 in Figure 2A. [Figure 4A] This figure shows an example of the communication network on which the mobile cleaning robot operates, and an example of data transmission within the network. [Figure 4B] This is a schematic diagram of the network. [Figure 5] This is a schematic diagram of the process of creating or improving an environment map. [Figure 6] This is a plan view of the environment map. [Figure 7] This is a plan view of the environment map. [Figure 8] This is a plan view of the environment map. [Figure 9] This is a plan view of the environment map. [Figure 10] This is a plan view of the environment map. [Figure 11] This is a plan view of the environment map. [Figure 12] This is a plan view of the environment map. [Figure 13] This is a plan view of the environment map. [Figure 14] This is a perspective view of the environment map. [Figure 15] This is a plan view of the environment map. [Figure 16] This is a plan view of the environment map. [Figure 17A] This is a plan view of the environment map. [Figure 17B] This is a plan view of the environment map. [Figure 18] A block diagram showing an example of a machine in which one or more embodiments may be implemented. [Modes for carrying out the invention]

[0010] FIG. 1 shows a plan view of a mobile cleaning robot 100 within an environment 40 according to at least one example of the present disclosure. The environment 40 can be a dwelling such as a house or an apartment and can include rooms 42a - 42e. Obstacles such as a bed 44, a table 46, and an island 48 can be placed within one or more of the rooms 42 of the environment. Each of the rooms 42a - 42e can respectively have a floor surface 50a - 50e. Some rooms such as room 42d can include a rug such as rug 52. The floor surface 50 can be of one or more types of flooring, such as hardwood, ceramic, low pile carpet, medium pile carpet, long (or high) pile carpet, stone, etc.

[0011] The mobile cleaning robot 100 can be operated by a user 60, etc., to autonomously clean the environment 40 room by room. In some examples, the robot 100 can clean the floor surface 50a of one room such as room 42a and then move to the next room such as room 42d and clean the surface of room 42d. Different rooms can have different types of floor surfaces. For example, room 42e (which can be a kitchen) may have a hard floor surface such as wood or ceramic tile, and room 42a (which can be a bedroom) may have a carpet surface such as medium pile carpet. Other rooms such as room 42d (which can be a dining room) may include multiple surfaces and a rug 52 is placed within room 42d.

[0012] During the cleaning operation or the ongoing operation, the robot 100 can create a map of the environment 40 using data collected from various sensors and calculations (such as distance measurement by odometry and obstacle detection). Once the map is created, the user 60 can define rooms or zones (such as room 42) within the map. The map can be presented to the user 60 on a user interface such as a mobile device, and the user 60 can indicate or change cleaning preferences.

[0013] During operation, the robot 100 can detect the surface type in each of the rooms 42, and the surface type can be stored in the robot or another device. The robot 100 can update the map (or related data) for including or considering the surface types of the floor surfaces 50a - 50e of each of the respective rooms 42 of the environment. In some examples, the map can be updated to indicate different surface types, such as in each of the rooms 42.

[0014] In some examples, the user 60 can define the action control zone 54, for example, using the methods and systems described herein. In response to the definition of the action control zone 54 by the user 60, the robot 100 can move towards the action control zone 54 to confirm the selection. After confirmation, the autonomous operation of the robot 100 can be started. In autonomous operation, the robot 100 can start an action in response to being within or near the action control zone 54. For example, the user 60 can define a dirty area of the environment 40 as the action control zone 54. In response, the robot 100 can start an intensive cleaning action, in which the robot 100 performs intensive cleaning of a portion of the floor surface 50d in the action control zone 54.

[0015] Components of the robot FIG. 2A shows a bottom view of the mobile cleaning robot 100. FIG. 2B shows an isometric view of the mobile cleaning robot 100. FIG. 3 shows a cross-sectional view of the mobile cleaning robot 100 taken along the indicator 3 - 3 of FIG. 2A. FIG. 3 also shows the front and rear orientation indicators. FIGS. 2A - 3 are described together below.

[0016] The cleaning robot 100 may be an autonomous cleaning robot capable of autonomously traveling across the floor surface 50 while sucking up debris 75 from different parts of the floor surface 50. As shown in Figures 2A and 3, the robot 100 may include a body 202 that is movable across the floor surface 50. The body 202 may include a plurality of connected structures on which the components of the cleaning robot 100 are mounted. The connected structures may include, for example, an outer housing for covering the internal components of the cleaning robot 100, a chassis or frame on which the drive wheels 210a and 210b and the cleaning rollers 205a and 205b (of the cleaning assembly 204) are mounted, and a bumper 238. The bumper 238 may be detachably fixed to the body 202 and may be movable relative to the body 202 while mounted to the body 202. In some examples, the bumper 238 may form part of the body 202.

[0017] As shown in Figure 2A, the body 202 includes a front section 202a having a substantially semicircular shape and a rear section 202b having a substantially semicircular shape. These sections may have other shapes, such as a rectangular front section (or a rectangular front section with rounded corners), in other examples. As shown in Figure 2A, the robot 100 may include a drive system including actuators 208a and 208b, which may be motors, for example. The actuators 208a and 208b may be mounted within the body 202 and operably connected to drive wheels 210a and 210b, which are rotatably mounted on the body 202 so that the body 202 can support itself on the floor surface 50. When driven, the actuators 208a and 208b can rotate the drive wheels 210a and 210b, enabling the robot 100 to move autonomously across the floor surface 50.

[0018] The controller (or processor) 212 may be located within a housing and may be a programmable controller, such as a single-board or multi-board computer, a direct digital controller (DDC), or a programmable logic controller (PLC). In other examples, the controller 212 may be any computing device, such as a handheld computer, for example, a smartphone, tablet, laptop, desktop computer, or any other computing device including a processor, memory, and communication functions. The memory 213 may be one or more types of memory, such as volatile or non-volatile memory, read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, and other storage devices and media. The memory 213 may be located within the body 202, connected to the controller 212, and accessible by the controller 212.

[0019] The controller 212 can operate actuators 208a and 208b to autonomously navigate the robot 100 around the floor surface 50 during cleaning. Actuators 208a and 208b may be operable to drive the robot 100 in the forward direction, in the reverse direction, or to change the direction of the robot 100. The robot 100 may include caster wheels 211 that can support the body 202 on the floor surface 50. The caster wheels 211 can support the front part 202a of the body 202 on the floor surface 50, and the drive wheels 210a and 210b can support the rear part 202b of the body 202 on the floor surface 50.

[0020] As shown in Figure 3, the vacuum assembly 218 may be located at least partially within the body 202 of the robot 100, for example, at the rear 202b of the body 202. The controller 212 can operate the vacuum assembly 218 to generate an airflow, which flows through a gap near the cleaning roller 205, through the body 202, and out of the body 202. The vacuum assembly 218 may include, for example, an impeller that generates the airflow when rotated. The airflow, and the rotating cleaning roller 205, can cooperate to draw the debris 75 into the intake duct 348 of the robot 100. The intake duct 348 may extend downward to the lower part of the body 202 or near it, and may be at least partially defined by the cleaning assembly 204.

[0021] The intake duct 348 may be connected to the cleaning head 204 or cleaning assembly, and may be connected to the cleaning bin 322. The cleaning bin 322 may be mounted inside the main body 202 and may contain debris 75 sucked in by the robot 100. A filter 349 may be located inside the main body 202 and may help separate the debris 75 from the airflow 220 before it enters the vacuum assembly 218 and is discharged from the main body 202. In this regard, the debris 75 may be captured in both the cleaning bin 322 and the filter before the airflow 220 is discharged from the main body 202.

[0022] The cleaning rollers 205a and 205b may be operably connected to one or more actuators 214a and 214b, for example, motors, respectively. The cleaning head 204 and the cleaning rollers 205a and 205b may be located in front of the cleaning bin 322. The cleaning rollers 205a and 205b may be mounted on the housing 224 of the cleaning head 204, and may also be mounted on the body 202 of the robot 100, for example, indirectly or directly. In detail, the cleaning rollers 205a and 205b may be mounted on the underside of the body 202 such that when the underside of the cleaning rollers 205a and 205b faces the floor surface 50 during cleaning, they engage with the debris 75 on the floor surface 50.

[0023] The housing 224 of the cleaning head 204 may be mounted on the body 202 of the robot 100. In this regard, the cleaning rollers 205a and 205b may also be mounted on the body 202 of the robot 100, such as indirectly through the housing 224. Alternatively or additionally, the cleaning head 204 may be a removable assembly of the robot 100, in which the housing 224 (in which the cleaning rollers 205a and 205b are mounted) is removablely mounted on the body 202 of the robot 100.

[0024] The side brush 242 may be connected to the underside of the robot 100 and to a motor 244 capable of operating to rotate the side brush 242 relative to the body 202 of the robot 100. The side brush 242 may be configured to engage with debris and move the debris toward the cleaning assembly 204 or away from the edge of the environment 40. The motor 244 configured to drive the side brush 242 may be in communication with the controller 212. The brush 242 may be a side brush that is laterally offset from the center of the robot 100 so that the brush 242 can extend beyond the outer circumference of the body 202 of the robot 100. Similarly, the brush 242 may also be offset forward from the center of the robot 100 so that the brush 242 can extend beyond the bumper 238 or the outer circumference of the body 202.

[0025] The robot 100 may further include a sensor system having one or more electrical sensors. The sensor system can generate one or more signals indicating the current location of the robot 100, and can generate one or more signals indicating the location of the robot 100 when the robot 100 is moving along the floor surface 50.

[0026] For example, a cliff sensor 234 (shown in Figure 2A) may be positioned along the bottom of the main body 202. The cliff sensor 234 may include an optical sensor, which may be configured to detect the presence or absence of an object below it, such as the floor surface 50. The cliff sensor 234 may be connected to a controller 212.

[0027] The collision sensors 239a and 239b (collision sensor 239) may be connected to the main body 202 and may be engaged with or configured to interact with the bumper 238. The collision sensor 239 may include a break beam sensor, a Hall effect sensor, a capacitive sensor, a switch, or other sensor capable of detecting contact between the robot 100 (e.g., the bumper 238) and an object in the environment 40. The collision sensor 239 may be communicating with the controller 212.

[0028] The image capture device 240 may be connected to the main body 202 and may extend at least partially through the bumper 238 of the robot 100, such as through an opening 243 in the bumper 238. The image capture device 240 may be a camera, such as a forward-facing camera, configured to generate signals based on images of the environment 40 of the robot 100. The image capture device 240 may transmit image capture signals to the controller 212 for use in navigation and cleaning routines.

[0029] The obstacle-following sensor 241 (shown in Figure 2B) may include an optical sensor directed outward or downward from the bumper 238, which may be configured to detect the presence or absence of an object adjacent to the side of the main body 202. The obstacle-following sensor 241 can emit a light beam horizontally in a direction perpendicular (or nearly perpendicular) to the forward driving direction of the robot 100. The optical emitter can emit the light beam outward from the robot 100, for example, horizontally, and the photodetector detects reflections of the light beam from objects near the robot 100. The robot 100 can determine the time of flight of the light beam, for example using the controller 212, and thereby determine the distance between the photodetector and the object, and therefore the distance between the robot 100 and the object.

[0030] Robot 100 may optionally include one or more dirt sensors 245 connected to the main body 202 and communicating with the controller 212. The dirt sensors 245 could be microphones, piezoelectric sensors, optical sensors, etc., and could be located in or near the debris flow path, such as near the opening of the cleaning roller 205 or within one or more ducts in the main body 202. This allows the dirt sensors 245 to detect how much dirt is being sucked in by the vacuum assembly 218 (e.g., via the extractor 204) at any given time during the cleaning mission. Because Robot 100 can recognize its location, it can log or record which areas or rooms on the map are dirtier or where more dirt is being collected. Robot 100 may also include a battery 245 capable of powering one or more components of the robot (such as motors).

[0031] Robot movements In some example operations, the robot 100 may be propelled in a forward or backward direction. The robot 100 may also be propelled so that it changes direction at a given position, or so that it changes direction while moving in a forward or backward direction.

[0032] When the controller 212 instructs the robot 100 to perform a mission, the controller 212 can operate the motor 208 to drive the drive wheels 210 and propel the robot 100 along the floor surface 50. In addition, the controller 212 can operate the motor 214 to rotate the rollers 205a and 205b, the motor 244 to rotate the brush 242, or the motor of the vacuum system 218 to generate airflow. The controller 212 can also execute software stored on the memory 213 to operate various motors or components of the robot 100, thereby causing the robot 100 to perform various navigation and cleaning actions.

[0033] Various sensors on the robot 100 can be used to help the robot navigate and clean within the environment 40. For example, a step sensor 234 can detect obstacles such as steep slopes and cliffs below the part of the robot 100 where the step sensor 234 is located. The step sensor 234 can send a signal to the controller 212 so that the controller 212 can change the direction of the robot 100 based on the signal from the step sensor 234.

[0034] In some examples, the collision sensor 239a may be used to detect the movement of the bumper 238 in one or more directions of the robot 100. For example, the collision sensor 239a may be used to detect the movement of the bumper 238 from front to rear, or the collision sensor 239b may be used to detect movement along one or more sides of the robot 100. The collision sensor 239 may send a signal to the controller 212 so that the controller 212 can change the direction of the robot 100 based on the signal from the collision sensor 239.

[0035] In some examples, the obstacle-following sensor 241 can detect detectable objects, including obstacles such as furniture, walls, people, and other objects in the environment of the robot 100. In some implementations, the sensor 241 may be positioned along the side of the body 202, and the obstacle-following sensor 241 can detect the presence or absence of objects adjacent to the side. One or more obstacle-following sensors 241 can also function as obstacle detection sensors similar to proximity sensors. The controller 212 can use signals from the obstacle-following sensors 241 to follow along obstacles such as walls or cabinets.

[0036] The robot 100 may also include sensors for tracking the distance traveled by the robot 100. For example, the sensor system may include an encoder associated with a motor 208 for a drive wheel 210, the encoder which can track the distance traveled by the robot 100. In some implementations, the sensors may include a light sensor facing downward toward the floor. The light sensor may be positioned to direct light through the bottom surface of the robot 100 toward the floor 50. The light sensor can detect the reflection of light and, based on the changes in the floor features as the robot 100 moves along the floor 50, can detect the distance traveled by the robot 100.

[0037] The image capture device 240 may be configured to generate signals based on images of the environment 40 surrounding the robot 100 as the robot 100 moves around the floor surface 50. The image capture device 240 may transmit such signals to the controller 212. The image capture device 240 may capture images of the walls of the environment so that features corresponding to objects on the walls can be used for localization.

[0038] The controller 212 can use data collected by the sensors of the sensor system to control the navigation behavior of the robot 100 during the mission. For example, the controller 212 can use sensor data collected by the robot 100's obstacle detection sensors (e.g., step sensor 234, collision sensor 239, and image capture device 240) to help the robot 100 avoid obstacles as it moves through its environment during the mission.

[0039] Sensor data can also be used by the controller 212 for simultaneous localization and mapping (SLAM) techniques, in which the controller 212 extracts or interprets environmental features represented by the sensor data and constructs a map of the floor surface 50 of the environment. Sensor data collected by the image capture device 240 can also be used for techniques such as visual-based SLAM (VSLAM), in which the controller 212 can extract visual features corresponding to objects in the environment 40 and use these visual features to construct a map. As the controller 212 guides the robot 100 around the floor surface 50 during the mission, the controller 212 can determine the location of the robot 100 on the map by using SLAM techniques to detect features represented in the collected sensor data and comparing those features with previously stored features. The map formed from the sensor data can show the locations of passable and impassable spaces in the environment. For example, the locations of obstacles may be shown on the map as impassable spaces, and the locations of open floor spaces may be shown on the map as passable spaces.

[0040] Sensor data collected by any of the sensors may be stored in memory 213. In addition, other data generated for SLAM techniques, including mapping data that forms a map, may be stored in memory 213. These data generated during the mission may include persistent data that is generated during the mission and available for use in subsequent missions. In addition to storing the software that causes the robot 100 to perform its actions, memory 213 can store data resulting from the processing of sensor data for access by the controller 212. For example, the map may be a map that is available and updatable from one mission to another by the robot 100's controller 212 to navigate the robot 100 around the floor surface 50.

[0041] Persistent data, including persistent maps, helps enable the robot 100 to efficiently clean the floor surface 50. For example, the map allows the controller 212 to guide the robot 100 towards open floor spaces and avoid impassable areas. In addition, for subsequent missions, the controller 212 can use the map to optimize the path the robot 100 will take during a mission, helping to plan its navigation through the environment 40.

[0042] Mapping example Figure 4A shows an example of a communication network 400 that can enable networking between a mobile robot 100 and one or more other devices, such as a mobile device 404, a cloud computing system 406, or another autonomous robot 408 separate from the mobile robot 404. Using the communication network 400, robot 100, mobile device 404, robot 408, and cloud computing system 406 can communicate with each other and send and receive data from one another. In some examples, robot 100, robot 408, or both robot 100 and robot 408 communicate with mobile device 404 through cloud computing system 406. Alternatively or additionally, robot 100, robot 408, or both robot 100 and robot 408 communicate directly with mobile device 404. Various types and combinations of wireless networks (e.g., Bluetooth, radio frequency, optical-based, etc.) and network architectures (e.g., mesh network) may be employed by the communication network 400.

[0043] In some examples, the mobile device 404 may be a remote device that can be linked to a cloud computing system 406 and allow a user to provide input. The mobile device 404 may include user input elements, such as one or more of a touchscreen display, buttons, a microphone, a mouse, a keyboard, or other devices that respond to input provided by the user. The mobile device 404 may also include an immersive medium (e.g., virtual reality) with which the user can interact to provide input. In these examples, the mobile device 404 may be a virtual reality headset or a head-mounted display.

[0044] The user can provide input corresponding to commands for the mobile robot 404. In such cases, the mobile device 404 can send a signal to the cloud computing system 406, which can then send command signals to the mobile robot 100. In some implementations, the mobile device 404 can present augmented reality images. In some implementations, the mobile device 404 may be a smartphone, laptop computer, tablet computing device, or other mobile device.

[0045] As described herein, the mobile device 404 may include a user interface configured to display a map of the robot environment. Robot paths, such as those identified by a coverage planner, may also be displayed on the map. The interface can receive user commands and modify the environment map, in particular by adding, removing, or otherwise changing no-go zones in the environment; adding, removing, or otherwise changing concentrated cleaning zones (such as areas requiring repeated cleaning) in the environment; restricting the robot's direction or pattern of passage in parts of the environment; or adding or changing cleaning ranks.

[0046] In some examples, the communication network 400 may include additional nodes. For example, the nodes of the communication network 400 may include additional robots. Also, the nodes of the communication network 400 may include network-connected devices that can generate information about the environment 40. Such network-connected devices may include one or more sensors, such as acoustic sensors, image capture systems, or other sensors that generate signals, to detect characteristics of the environment 40 from which features can be extracted. Network-connected devices may also include home cameras, smart sensors, etc.

[0047] In communication network 400, wireless links can utilize various communication methods and protocols, such as Bluetooth Class, WiFi, Bluetooth Low Energy (also known as BLE), 802.15.4, Worldwide Interoperability for Microwave Access (WiMAX), infrared channels, and satellite bands. In some examples, wireless links may include any cellular network standard used to communicate between mobile devices, including, but not limited to, standards that qualify as 1G, 2G, 3G, 4G, 5G, etc. Network standards qualify as one or more generations of mobile telecommunications standards by meeting specifications or standards, such as those maintained by the International Telecommunication Union, when used. For example, a 4G standard may correspond to the International Mobile Telecommunications Advanced (IMT Advanced) specification. Examples of cellular network standards include AMPS, GSM, GPRS, UMTS, LTE, LTE Advanced, Mobile WiMAX, and WiMAX Advanced. Cellular network standards can use various channel access methods, such as FDMA, TDMA, CDMA, or SDMA.

[0048] Figure 4B shows an exemplary process 401 for exchanging information between devices in a communication network 400, including a mobile robot 100, a cloud computing system 406, and a mobile device 404.

[0049] In some example operations, a cleaning mission can be started by pressing a button on the mobile robot 100 (or mobile device 404) or scheduled for a future time or date. The user can select a set of rooms to be cleaned during the cleaning mission, or can instruct the robot to clean all rooms. The user can also select a set of cleaning parameters to be used in each room during the cleaning mission.

[0050] During a cleaning mission, the mobile robot 100 can perform various navigation processes, including data collection such as obstacle detection and avoidance (ODOA) and visual scene understanding (VSU), as it travels through the environment at 410. The robot 100 can also perform visual simultaneous localization and mapping (VSLAM). VSLAM may be performed by the robot 100 using an optical stream generated by the image capture device 240 and may be used by the robot 100 to compare features detected frame by frame in order to build or update a map of its environment (environment 40, etc.) and to determine the position of the robot 100 within the environment. ODOA may be performed using an optical stream and may be used to detect obstacles in the robot's path so that ODOA analysis can see objects below the horizon and as close as possible to the front of the robot. VSU may be performed by analyzing an optical stream using all or most of the frame for understanding or interpreting objects in the environment, and may also be used for mapping or localization. Robot 100 may also collect data from other sensors, such as a collision sensor 239, an obstacle-following sensor 241, or any other sensors on robot 100.

[0051] Data from one or more processes (or operations of robot 100) may be stored by robot 100, cloud computing system 406, or mobile device 404. These data, components, or processes may be used together to perform one or more cleaning missions, the missions may be modifiable by the user, and may be used in other processes performed by mobile device 404 (or its processor 432) or cloud computing system 406. For example, in 410, robot 100 may use one or more optical processes to determine whether an object is placed in the environment. For example, the robot may use VSU to determine that an object is present in the environment, and use one or more of VO, ODOA, or VSLAM to determine the location of the object in the environment and on a map. Robot 100 may use the determined location of the object to place the object in the environment. Robot 100 may also refer to its geographic location. The robot 100 can use its geographical location to more accurately position detected objects, or it can store the robot's location data for further analysis of detected objects.

[0052] At 411, the mobile robot 100 can transfer and store data (e.g., one or more of location data, motion event data, time data, etc.), object detection, and object locations within the robot 100. At 412, the robot 100 can collect, store, or analyze sensor data (or analysis performed by the robot 100), as will be described in more detail below. At 414, the robot 100 can use this information (optionally together with other information) to generate or update a map of the environment using a processor 212, etc., which may involve various steps, as will be described in more detail below. Optionally, the cloud computing system 406 can perform one or more of steps 412 and 414.

[0053] The map may be transmitted to the mobile device 404 in 416, and the mobile device 404 may display the map on its screen or elsewhere in 418. User 405 can view the map in 420 and modify the map in 422 using one or more interfaces of the mobile device 404 (e.g., a touchscreen). The map may be updated in 424 by user 405 or the mobile device 404, and transmitted in 426 to the cloud computing system 406 or the robot 100, so that the robot 100 can use the updated map when performing one or more missions or actions in 428. Optionally, the map may not be modified by the user in 414 and may be used in 427 by the robot 100 to perform one or more missions or actions in 428.

[0054] The operations for process 401 and other processes described herein, such as one or more steps described below, may be performed in a distributed manner. For example, the cloud computing system 406, the mobile robot 100, and the mobile device 404 may cooperate with each other to perform one or more of the operations. In some implementations, the operations described as being performed by one of the cloud computing system 406, the mobile robot 100, and the mobile device 404 may be performed at least partially by two or all of the cloud computing system 406, the mobile robot 100, and the mobile device 404.

[0055] Figure 5 shows a schematic diagram of Method 500, according to at least one example of the present disclosure. Method 500 may be a method for changing the boundaries of an environment map (or creating an environment map). The steps or operations of Method 500 are shown in a particular order for convenience and clarity, and many of the operations described may be performed in different sequences or in parallel without substantially affecting other operations. The steps or operations of Method 500 may be omitted or performed multiple times. Method 500 described includes operations that may be performed by multiple different actors, devices, and / or systems. It should be understood that a subset of the operations described in Method 500 that may be attributable to a single actor, device, or system may be considered a separate, independent process or method.

[0056] Method 500 may be performed by one or more of the robot 100, the cloud computing system 406, or the mobile device 404, and may include, or be part of, one or more of the steps of process 401 described above, such as step 412 or 414. Method 500 may begin in step 502, in which sensor data may be received by the robot 100, the mobile device 404, or the cloud computing system 406, etc. The sensor data may include data from one or more of the robot 100's sensors based on interaction between the mobile cleaning robot 100 and the environment 40.

[0057] In step 504, obstacles may be defined by using sensor data, such as light or collision sensor data, to generate boundaries of spaces that the mobile cleaning robot can pass through in the environment. The boundaries can optionally define at least partially spaces in the environment that the robot 100 cannot pass through at a height from the floor of the environment. The impassable spaces may include spaces beyond the boundaries that are observed by the mobile cleaning robot but are not traveled through by the mobile cleaning robot. In step 506, an image or map may be generated, and the image may represent one or more obstacles or boundaries in or within the environment. In 508, the image may be further modified to generate modified boundaries of the environment using impassable spaces and sensor data, such as by inserting or moving walls, or otherwise dividing or subdividing spaces within the modified boundaries. A map of the environment may be generated at least partially based on the boundaries or modified boundaries.

[0058] In step 510, the room may be optimized. For example, the map or space may be segmented or divided into rooms defined by the room boundary, based at least partially on the modified boundary or at least partially on the impassable space. The area obtained from the divided negative space may be assigned to the room based on relevant visual features observed from the space inside the room. In step 512, objects within a space, such as an impassable space, or within a portion of that space, may be characterized. For example, based on sensor data and the map, the space or object may be characterized as a wall or other fixed object. Optionally, the room boundary may be modified at least partially on the characterized portion of the impassable space.

[0059] Such methods can be computationally efficient because they do not require expensive or complex models to recognize or define negative space as objects. Therefore, robot 100 could be used to build or assemble more accurate maps using relatively inexpensive sensors and processors.

[0060] In some examples, an object or space may be characterized in three dimensions. For example, the height of a characterized portion may be determined using data or images from robot 100, such as data or images showing the corners or lines of the object. Then, a three-dimensional representation of each characterized portion may be generated (for example, by a mobile device 404 or a cloud computing system 406) using a map and the height of the characterized portion. A three-dimensional map may be generated based at least on the map and on the three-dimensional representation of the characterized portion. In step 514, a map may be displayed that may include any of the versions of the map described above.

[0061] Figure 6 shows a plan view of a map 600 of an environment such as environment 40. Map 600 may be a preliminary map or visualization of a collection of data points collected by the robot 100. For example, each object 636 (of objects 636a to 636n) may be a part of the environment that is impassable by the robot 100, and can at least partially define a preliminary boundary of space or environment. Whether each part is impassable may be determined by comparing each object or part recorded or stored by the robot 100, the cloud computing system 406, or the mobile device 404 with each part that the robot 100 has traveled through at some point in one or more missions. In other words, each object or part 636 may be a part of the environment 40 that is not traveled by the robot 100 but has been captured through one or more events or interactions by the robot 100, such as a collision indicated by a collision sensor, or an image captured by an image capture device or LiDAR (light detection and ranging). Optionally, each part 636 may be an interaction (or lack thereof).

[0062] Figure 7 shows a plan view of the environment map 700. Map 700 may be a modified map or a visualization of a map generated based on data points shown in map 600. A robot 100, a mobile device 404, or a cloud computing system 406 (or another device) can use sensor data to generate boundaries of spaces passable by a mobile cleaning robot within the environment 40, where the boundaries at least partially define impassable spaces in the environment, and the impassable spaces include areas that extend beyond the boundaries. Figure 7 shows how a device (or system) may segment map 700 into rooms defined by room boundaries, at least partially based on modified boundaries or impassable spaces.

[0063] More specifically, a robot 100, a mobile device 404, or a cloud computing system 406 can use one or more objects or parts 636 of the environment 40 (map 600) to generate a boundary 740 of the environment 40. The boundary 740 may include one or more room boundaries 742, such as room boundaries 742a to 742n. A room boundary 742 may independently or collectively define one or more rooms 744, such as rooms 744a to 744n, and may be an example of a modified boundary of an environment or space. For example, a room boundary 742a may define room 744a at least partially, and a room boundary 742b may define room 744b.

[0064] Room boundaries 742 may be placed within map 700 based on objects or portions 636 of map 600 that are impassable, so that room boundaries 742 may generally be relatively straight lines inferred or determined from impassable portions 636. One or more rooms 744 may be determined or placed based on room boundaries 742 and openings 746. Openings 746 may be inferred (based on portions 636 or their absence, etc.) to be doorways or passages between rooms. Such maps may be presented to a user 405 via a mobile device 404, etc., or may be further modified as described below.

[0065] Figure 8 shows a plan view of the environment map 700. Map 700 may be consistent with the map 700 described with respect to Figure 7, and Figure 8 shows how map 700 may be provisionally modified. More specifically, Figure 8 shows that boundary lines 740 may be modified by moving them around the space and forming something close to the outer walls of the environment or the outer walls of a room. Boundary 740 may be an example of a modified boundary or border of a space or environment.

[0066] Figure 8 also shows that map 700 can be segmented into rooms 744 defined by room boundaries 742. The placement of boundaries 742 can be determined at least partially on modified boundaries 748 or at least on impassable spaces, such as by or determined from the interaction between robot 100 and the environment (e.g., part 636). The shape of room 744a can be reshaped based on impassable spaces, etc., by placing outer walls 748a around passable spaces 750a and impassable spaces 752a of room 744a. Rooms 744b-744n can similarly be modified to redefine the boundaries 740 of map 700, helping to create a more accurate, more recognizable, or more realistic version of map 700 for presentation to the user.

[0067] Boundary 748 may be determined by placing one or more polygons to represent a space that is aligned with or substantially aligned with portion 636. Once polygons are placed, one or more systems or devices may determine whether the polygons define a valid area or room relative to other areas or rooms in the map or environment, by using one or more constraints or rules, such as rectilinear analysis, nearly convex segmentation, shape grammar analysis, room aspect ratio analysis, corridor aspect ratio analysis, or grouping of adjacent rooms. Optionally, an evaluator may be used to assess the effectiveness of a polygon defining an area or room, and the evaluator may provide a score to define how closely the polygon defines a room.

[0068] Optionally, one or more systems or devices can determine whether a polygon defines an invalid area or room by using one or more constraints or rules, such as room segmentation based on the mid-axis of available space or random segmentation of rooms or the environment, using substantially convex segmentation, visible space segmentation (segmenting the visible space of the environment), information-based segmentation (e.g., based on floor type, room type, threshold, etc.), etc. If a polygon satisfies the rules or analysis for a valid area and is determined not to be an invalid area, the polygon may be set as a modified boundary (e.g., modified boundary 748a).

[0069] Once a single valid polygon or modified boundary 748 is placed, valid modified boundaries may be used to place additional modified boundaries. A set of actions may be used to create a valid modified boundary or polygon from another valid modified boundary or polygon, such as by shifting the edges of a boundary adjacent to a valid boundary, or by moving one or more vertices of a boundary adjacent to a valid boundary. For example, once boundary 748a is determined to be valid, the edges of boundary 748b may be modified or moved (normally or parallel, for example) or the vertices of boundary 748b may be modified or moved, for example, to better align with boundary 748a.

[0070] Once boundary 748 and room boundary 742 are established, they can be analyzed for their suitability within a space or environment. For example, boundary 748 may be determined to define a room or area, and that room or area may be compared with other data or analysis to determine how well boundary 748 and boundary 742 represent the room or space. For example, boundary 748a or room 744a may be compared with map 600 (or its data) to determine the suitability of boundary 748a or room 744a.

[0071] Following the fit analysis (or any other step), the boundary 748 may be improved to enhance the fit between the generated floor plan and the raw occupancy map 600 in Figure 6, using, optionally, one or more of the processes described above or, with respect to Figures 9-16, below. Optionally, the boundary 748 may be improved in other ways. For example, the boundary 748 may be optimized using dynamic analysis, such as good old analysis. The boundary 748 may also be optimized using gradient-based analysis, stochastic optimizers, population-based metaheuristics, Monte Carlo tree search, hidden Markov models, and generative adversarial networks. Following optimization (or any other step described above), the boundary 748 may be finalized, and the map 700 may be presented to the user 405 via a mobile device 404, etc., or may be further modified as described below.

[0072] Figure 9 shows a plan view of the environment map 900. Map 900 may be similar to maps 600-700 described above with respect to Figures 6-8. Figure 9 shows the same environment derived from different environments or different data, illustrating how the map may be further modified following preliminary changes to its boundaries.

[0073] Map 900 may include boundary lines 940, at least partially defined by boundaries 942a-942n, which can separate traversable space 950 from impassable space 952. Boundaries 942a-942n may be used to at least partially define rooms 944a-944n. Map 900 may be updated to include data or images (such as points or lines) representing visual features or objects that are within the environment but captured outside traversable space 950. For example, object 954a may be outside traversable space 950a of room 944b (across room boundary line 942a) but located within impassable space 952a adjacent to room 944a. Features 954a-954n may similarly be overlaid (or otherwise used) on the map to further modify map 900 and boundary lines 940.

[0074] Feature 954 may be generated or made to occur from one or more data points, or from an analysis of one or more data points, based on sensor data from robot 100, etc. For example, feature 954 may be derived from one or more of the following: SLAM data, VSLAM data, ODOA data, VSU data, LiDAR, etc. Feature 954 may be used to further modify map 900, as will be described in more detail below.

[0075] Figure 10 shows a plan view of the environment map 900. Map 900 may be derived from or include map 900 in Figure 9. Figure 10 shows additional data and modifications to map 900. For example, Figure 10 shows that boundary line 940 may be overlaid on map 900, which may be derived from the steps described above with respect to Figures 6-7, or may be derived based on boundary line 940.

[0076] Map 900 may also include features 954a-954n located within impassable space 952 of the environment (or outside passable space 950). Figure 10 also shows the positions or poses 956a-956n of the robot 100. Each pose 956 shown on map 900 may represent a location on map 900 where the robot 100 is, or was, positioned when feature 954 outside passable space 950 was collected. For example, pose 956a may represent the robot's location in the environment when feature 954a was recorded.

[0077] Poses 956 and features 954 can be grouped based on the rooms in map 900, such as based on which room pose 956 was recorded in and which room features 954 were near (or relatively near). For example, all features 954 (e.g., 954a) associated with pose 956 (e.g., 956a) occurring in room 944a can be grouped together, and all features 954 (e.g., 954b) associated with pose 956 (e.g., 956b) occurring in room 944b can be grouped together. Such features can be represented on 900 in different or distinct colors. The boundaries 940 of each room can be adjusted or modified based on one or more groupings of poses 956 or features 954. For example, the boundaries 940 can be adjusted from the boundaries 940 created or generated in the steps or procedures (e.g., 740) of Figures 6–8. The boundaries 940 can also be further modified, as described with respect to Figure 11.

[0078] Figure 11 shows a plan view of the environment map 900. Map 900 may be consistent with map 900 in Figure 10. Figure 11 shows how the map may be further modified. For example, Figure 11 shows how the boundary lines 940 may be modified based on features and locations.

[0079] A room boundary 958 may be an example of a modified boundary or border of a space or environment. More specifically, Figure 11 shows that each room boundary 958 may be modified based on features 954 and poses 956. For example, room boundary 958a may be modified from room boundary 942a based at least partially on objects 954a and poses 956a associated with room 944a, and room boundary 958b may be modified from room boundary 942b based at least partially on objects 954b and poses 956b associated with room 944b. Other room boundaries 958 may be modified similarly based on their associated features 954 and poses 956.

[0080] As shown in Figure 11, room boundary lines 958d and 958n may be increased from room boundary lines 942d and 942n, respectively, to include the walls of each room within each room. For example, the gap between boundary lines 942d and 942n may be allocated to room 944d, and boundary line 958d may be moved accordingly.

[0081] Optionally, the height of each feature 954 may be taken into consideration (for example, by a robot 100, a mobile device 404, or a cloud computing system 406) to determine which object 954 is a wall, and thus can be used to determine where each of the room boundaries 958 should be placed. Also, room boundaries 958, such as room boundary 958a and room boundary 958b, may overlap when modified. Such overlap may indicate that the wall dividing rooms 944a and 944b is located on or between the shared overlapping room boundaries.

[0082] As also shown in Figure 11, the room boundary 958 may be modified or moved to include more of the objects 954 from impassable spaces 952 within the room. For example, the height of an object or feature 954 may be considered (e.g., by a robot 100, a mobile device 404, or a cloud computing system 406) to determine which objects 954 should be incorporated into the room 944 rather than a wall (or window), such as furniture (e.g., a bed or bookshelf). For example, Figure 11 shows that the room boundary 958d of room 944d may be expanded to include features or objects 954d that may have a height less than the perceived height of a wall or ceiling, and that object 954d represents an object in room 944d, such as a furniture item. The boundary 958 may be further analyzed (for fitting or optimization, etc.) using one or more of the steps or processes described above with respect to Figures 6–8.

[0083] As will be described in more detail below, each of the objects 954 (or groups of objects or features, or negative space) may be grouped or classified by the robot 100, the mobile device 404, or the cloud computing system 406. For example, objects or areas in inaccessible space may be characterized as walls (long, internal, external), windows, inaccessible rooms, built-in items (short, large, fixed items such as large furniture, cabinets, and equipment), or clutter (narrow spaces, limited variation, such as toy boxes and kitchen chairs). Objects may also be classified as fixed objects or dynamic or obstructive objects (objects moving within the space), or clutter. Such classifications may be used further in other ways to modify room boundaries 958 or maps 900.

[0084] Figure 12 shows a plan view of the environment map 1200. Figure 13 shows a plan view of the environment map 1200. Figures 12 and 13 are described together below.

[0085] Map 1200 may be similar to the map described above. Map 1200 may represent a different environment. Figure 12 shows Map 1200 in which boundary lines 1240 include modified boundary lines 1258 (e.g., 1258a-1258d) associated with each room 1244a-1244d. Figure 12 also shows negative space (or impassable space) 1252 which may include passable space 1250 (e.g., 1250a-1250d) associated with each room and impassable space (e.g., 1252a-1252d) associated with each room. Map 1200 may represent a map in which boundary lines are adjusted based on the height of objects or features within them, or adjusted in other ways as described above. For example, modified boundary lines 1258 may be derived at least in part based on data from robot 100, including heights determined from map 900 or objects in the environment (e.g., feature 954).

[0086] Figure 12 shows the impassable spaces 1252 arranged within each room, and Figure 13 shows the map 1200 with the boundary lines 1240 (e.g., modified boundary line 1258) removed, showing the impassable spaces 1252a to 1252d as separate or individual objects. Figure 13 also shows an example of how objects may be categorized. For example, a device or system such as a robot 100, a mobile device 404, or a cloud computing system 406 can use data from the robot 100 to determine the characteristics of each section of the impassable space 1252, such as one or more objects or features (e.g., feature 954).

[0087] For example, features (e.g., feature 954) may be used to determine the height, width, or depth of each object (in some examples, before the object's identification information can be determined), including whether the object is flush with the floor or ceiling, or whether there is a gap between them. These determinations may be made by a device or system and may be used to characterize each portion of the impassable space 1252. For example, impassable space 1252a may be determined to be an object by determining that impassable space 1252a has a height lower than the height of a wall or ceiling. Similarly, impassable spaces 1252d and 1252e may be determined to be objects. Impassable space 1252b may be determined to be a wall by determining that impassable space 1252b has a height common to another wall or ceiling. Impassable space 1252c may be determined to be an inaccessible space based on determining that the robot 100 cannot access the space or that its dimensions cannot be determined. As robot 100 collects more observations from subsequent missions, more areas of negative or impassable space can be better recognized and classified. Once each portion of impassable space 1252 is characterized, the boundary lines 1240 may be updated (as described below, for example), and impassable space 1252 may be further modified to produce a more accurate representation of map 1200, as described below.

[0088] Figure 14 shows a perspective view of the environment map 1400. Map 1400 may be similar to the map described above. Map 1400 may represent a different environment. Map 1400 may represent a map in which impassable spaces are characterized or categorized as described above. Figure 14 shows map 1400 in which impassable objects and walls 1452 are represented in three dimensions.

[0089] More specifically, map 1400 may include rooms 1444 (e.g., rooms 1444a to 1444n) with modified boundaries 1458 (e.g., boundaries 1458a to 1458n) that may include impassable spaces 1452 (e.g., impassable spaces or objects 1452a to 1452n). Map 1400 may also include openings 1446 that may be doors or entrances between rooms. Figure 14 shows map 1400 with impassable objects and walls 1452 represented in three dimensions.

[0090] Once an item is characterized at least partially based on the dimensions of an object (as described above with respect to Figures 6-13), objects or impassable spaces 1452 may be represented based on inference or characterization. For example, walls 1458 (e.g., wall 1458a) may all be the same or similar height and may be shown in a common color. Other impassable spaces 1452 or objects may be shown to have their determined height or height relative to the walls 1458. For example, an impassable space or object 1452c may have a height lower or less than the height of a wall to indicate the presence of a bed or table.

[0091] Optionally, impassable spaces 1452 or objects in each room may be indicated by a common color by the room to help distinguish them between spaces or rooms in the environment. Openings 1446 may also be represented as having a reduced height relative to walls or boundaries 1458. In this way, a three-dimensional representation of the environment may be presented as a map 1400, thereby helping the user more easily recognize the mapped space and helping the user more easily modify and customize the map for robot operation and missions.

[0092] Figure 15 shows a plan view of the environment map 1500. Figure 16 shows a plan view of the environment map 1500. Figures 15 and 16 are described together below. Map 1500 may be similar to any of the maps described above, so that map 1500 may include room 1544 (for example, room 1544a). Figures 15 and 16 show how the maps can be modified as needed.

[0093] For example, Figure 15 shows a map 1500 including boundary lines 1558, which may be determined based on any of the methods or steps described above. Figure 16 shows how boundary lines 1558 may be modified so that the exterior walls 1560 are represented by relatively thick lines or boundary lines and the interior walls 1562 are represented by relatively thin lines or boundary lines, so that when presented to a user it may appear as a higher quality floor plan or map that can help the user recognize the environment more easily.

[0094] To determine which walls are interior and exterior walls, the robot 100, mobile device 404, or cloud computing system 406 can use data from the robot 100, such as sensor data, to determine which walls are common walls and which are not shared or common. One or more of the devices or systems can also determine which walls are located on or near the perimeter of the passable space 1550, which walls are located near or beyond the perimeter of the passable space, or which walls are located in or near the impassable space.

[0095] Figure 17A shows a plan view of the environment map 1700. Figure 17B shows a plan view of the environment map 1700. Map 1700 may be similar to any of the maps described above, so that map 1700 may include room 1744. Figures 17A and 17B show how the map can be modified as needed.

[0096] More specifically, a robot 100, a mobile device 404, or a cloud computing system 406 can generate a boundary 1740 of the environment 40 using one or more objects or parts 636 of the environment 40 (map 600), as shown in Figure 17A. The boundary 1740 may be or include a room boundary 1742. The boundary 1740 may be at least partially defined by an impassable space 1752 and may include a passable space 1750. In Figure 17A, the impassable space 1752 may cause the boundary 1740 to appear as a relatively unrecognizable shape, for example, a U-shape, because, prior to its modification, a relatively large portion of the room 1744 is occupied by the impassable space 1752, which may be a bed, a desk, or other relatively large object. One or more of the methods or steps described above may be used to modify the map 1700.

[0097] Figure 17B shows how boundary 1740 can be modified to represent impassable space 1752 using a polygon, in order to generate a modified boundary 1758 and define a modified boundary 1748 for impassable space 1752. For example, Figure 17B shows how the modified boundary 1758 can better represent a room with a bed, which may be more recognizable to the user than the U-shaped space in Figure 17A.

[0098] Figure 18 shows a block diagram of an exemplary machine 1800 in which any one or more of the techniques (e.g., methods) described herein may be performed. The examples described herein may include, or be operated by, logic or several components or mechanisms in machine 1800. A circuit (e.g., a processing circuit) is a set of circuits implemented in the tangible entities of machine 1800, including hardware (e.g., simple circuits, gates, logic, etc.). The elements of a circuit may be flexible over time. A circuit includes elements that, when operating, can perform a specified operation individually or in combination. In one example, the hardware of a circuit may be designed immutably (e.g., hardwired) to perform a particular operation. In one example, the hardware of a circuit may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a machine-readable medium that has been physically modified (e.g., magnetically, electrically, such as a movable arrangement of invariant mass particles) to encode instructions for a particular operation. In the connection of physical components, the fundamental electrical properties of the hardware components are changed, for example, from insulator to conductor, or vice versa. Instructions enable embedded hardware (e.g., an execution unit or loading mechanism) to create elements of a circuit in the hardware via variable connections to perform a specific part of operation while in operation. Thus, in one example, a machine-readable medium element is part of a circuit or is communicatively coupled to other components of a circuit when the device is in operation. In one example, any of the physical components may be used in two or more elements of two or more circuits. For example, during operation, an execution unit may, at one point in time, be used in a first circuit of a first circuit configuration, and at a different time, be reused by a second circuit in the first circuit configuration, or by a third circuit in the second circuit configuration. Additional examples of these components with respect to machine 1800 follow.

[0099] In alternative embodiments, machine 1800 may operate as a standalone device or may be connected to other machines (e.g., networked). In a networked deployment, machine 1800 may operate as a server machine, a client machine, or both in a server-client network environment. For example, machine 1800 may act as a peer machine in a peer-to-peer (P2P) (or other distributed) network environment. Machine 1800 may be a personal computer (PC), tablet PC, set-top box (STB), personal digital assistant (PDA), mobile phone, web device, network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) that specify the actions to be taken by that machine. Furthermore, although only a single machine is shown, the term “machine” shall also be interpreted to include any set of machines that individually or collectively execute a set of instructions (or sets of instructions) to perform any one or more of the methods described herein, such as cloud computing, software as a service (SaaS), and other computer cluster configurations.

[0100] The machine (e.g., a computer system) 1800 may include a hardware processor 1802 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), main memory 1804, static memory (e.g., memory or storage for firmware, microcode, basic input / output (BIOS), Unified Extensible Firmware Interface (UEFI), etc.) 1806, and mass storage 1816 (e.g., a hard drive, tape drive, flash storage, or other block device), some or all of which may communicate with each other via an interlink (e.g., a bus) 1808. The machine 1800 may further include a display unit 1810, an alphanumeric input device 1812 (e.g., a keyboard), and a user interface (UI) navigation device 1814 (e.g., a mouse). In one example, the display unit 1810, the input device 1812, and the UI navigation device 1814 may be touchscreen displays. The machine 1800 may further include a storage device (e.g., a drive unit) 1816, a signal generating device 1818 (e.g., a speaker), a network interface device 1820, and one or more sensors 1828 such as a Global Positioning System (GPS) sensor, a compass, an accelerometer, or other sensors. The machine 1800 may also include an output controller 1830 for communicating with or controlling one or more peripheral devices (e.g., a printer, a card reader, etc.) via a series (e.g., Universal Serial Bus (USB)), parallel, or other wired or wireless (e.g., infrared (IR), near-field communication (NFC), etc.) connection.

[0101] The registers of processor 1802, main memory 1804, static memory 1806, or mass storage 1816 may be or include a machine-readable medium 1822 in which one or more sets of data structures or instructions 1824 (e.g., software) are stored, which embody or utilize any one or more of the techniques or functions described herein. The instructions 1824 may also be entirely or at least partially present in any of the registers of processor 1802, main memory 1804, static memory 1806, or mass storage 1816 while they are being executed by machine 1800. In one example, one or any combination of the hardware processor 1802, main memory 1804, static memory 1806, or mass storage 1816 may constitute the machine-readable medium 1822. Although machine-readable medium 1822 is shown as a single medium, the term “machine-readable medium” may include a single or multiple mediums configured to store one or more instructions 1824 (for example, a centralized or distributed database, and / or associated caches and servers).

[0102] The term “machine-readable medium” can include any medium capable of causing the machine 1800 to execute any one or more of the techniques of the present disclosure, storing, encoding, or carrying instructions for execution by the machine 1800, or storing, encoding, or carrying data structures used by or associated with such instructions. Examples of non-limiting machine-readable mediums may include solid-state memory, optical mediums, magnetic mediums, and signals (e.g., radio frequency signals, other photon-based signals, audio signals, etc.). In one example, a non-temporary machine-readable medium comprises a machine-readable medium using a plurality of particles having invariant (e.g., stationary) mass, and is therefore a composition. Thus, a non-temporary machine-readable medium is a machine-readable medium that does not contain a transient propagating signal. Specific examples of non-temporary machine-readable media may include non-volatile memory such as semiconductor memory devices (e.g., electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM)) and flash memory devices, magnetic disks such as internal hard disks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks.

[0103] Instruction 1824 may be further transmitted or received over a communication network 1826 using a transmission medium via a network interface device 1820 that utilizes one of several transport protocols (e.g., Frame Relay, Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), Hypertext Transfer Protocol (HTTP), etc.). Illustrative communication networks may include, in particular, local area networks (LANs), wide area networks (WANs), packet data networks (e.g., the Internet), mobile phone networks (e.g., cellular networks), traditional telephone (POTS) networks, and wireless data networks (e.g., the IEEE 802.11 standard family known as WiFi®, the IEEE 802.16 standard family known as WiMAX®), the IEEE 802.15.4 standard family, and peer-to-peer (P2P) networks. For example, the network interface device 1820 may include one or more physical jacks (e.g., Ethernet jacks, coaxial jacks, or telephone jacks) or one or more antennas for connecting to the communication network 1826. For example, the network interface device 1820 may include multiple antennas for wireless communication using at least one of the following techniques: single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO). The term “transmission medium” is to be interpreted as including any intangible medium capable of storing, encoding, or carrying instructions for execution by the machine 1800, and including digital or analog communication signals or other intangible mediums to facilitate the communication of such software. The transmission medium is a machine-readable medium.

[0104] Notes and examples The following non-limiting examples detail several aspects of this subject matter, particularly in order to solve the problem and provide the benefits described herein.

[0105] Example 1 is at least one non-temporary machine-readable medium which, when executed, includes instructions causing a processing circuit to perform an action, the action of receiving sensor data from a mobile cleaning robot based on interaction between the mobile cleaning robot and the environment, and using the sensor data to generate boundaries of spaces that the mobile cleaning robot can pass through in the environment, the boundaries of which define at least partially impassable spaces in the environment, the impassable spaces including areas beyond the boundaries, and generating modified boundaries of the environment using the impassable spaces and sensor data.

[0106] In Example 2, it is optionally included that the subject of Example 1 is to cause the processing circuit to perform further actions to generate a map of the environment based on the modified boundaries.

[0107] In Example 3, the subject of Example 2 optionally includes the fact that the instruction causes the processing circuit to perform an operation to segment the map into rooms defined by the room boundaries, at least partially based on the modified boundaries.

[0108] In Example 4, the subject of Example 3 optionally includes the fact that the map is segmented into rooms based at least partially on impassable spaces.

[0109] In Example 5, the subject of Example 4 optionally includes the fact that the instruction causes the processing circuit to perform an action to characterize the portion of the space that is impassable using sensor data.

[0110] In Example 6, the subject of Example 5 optionally includes the fact that the instruction causes the processing circuit to perform an action to modify the room boundary, at least partially based on the characterized portion of the impassable space.

[0111] In Example 7, the subject of Example 6 optionally includes the fact that a portion is characterized using images captured by an image capture device of a mobile cleaning robot.

[0112] In Example 8, the subject of Example 7 optionally includes the fact that the images are captured using the VSLAM process.

[0113] In Example 9, any one or more of the subjects from Examples 7 to 8 optionally include causing a processing circuit to perform the following actions: using an image to determine the height of a featured portion, and using the map and the height of the featured portion to generate a three-dimensional representation for each of the featured portions.

[0114] In Example 10, the subject of Example 9 optionally includes the fact that the instruction causes the processing circuit to perform an operation to generate a three-dimensional map based on the map and based on the three-dimensional representation of the characterized parts.

[0115] In Example 11, any one or more subjects from Examples 1 to 10 optionally include a boundary-crossing space where the impassable space is observed by and not traveled by the mobile cleaning robot.

[0116] Example 12 is a method for generating an environment map using a mobile cleaning robot, the method comprising: receiving sensor data based on interaction between the mobile cleaning robot and the environment; using the sensor data to generate boundaries of spaces that the mobile cleaning robot can pass through in the environment, wherein the boundaries define at least partially the spaces that the environment cannot pass through; using the spaces that cannot pass through and the sensor data to generate modified boundaries of the environment; and generating an environment map based on the modified boundaries.

[0117] In Example 13, the subject of Example 12 optionally includes a boundary-crossing space where the impassable space is observed by the mobile cleaning robot and is not traveled by the mobile cleaning robot.

[0118] In Example 14, any one or more themes from Examples 12 to 13 optionally include the step of segmenting the map into rooms defined by room boundaries, at least partially based on the modified boundaries.

[0119] In Example 15, the subject of Example 14 optionally includes the fact that the map is segmented into rooms based at least partially on impassable spaces.

[0120] In Example 16, the subject of Example 15 optionally includes the step of using sensor data to characterize portions of impassable space.

[0121] In Example 17, the subject of Example 16 optionally includes the step of altering the room boundary, at least partially based on the characterized portion of the impassable space.

[0122] In Example 18, the subject of Example 17 optionally includes the steps of: determining the height of the characterized parts using an image; generating a three-dimensional representation for each of the characterized parts using the map and the height of the characterized parts; and generating a three-dimensional map based on the map and based on the three-dimensional representation of the characterized parts.

[0123] Example 19 is at least one non-temporary machine-readable medium which, when executed, includes instructions causing a processing circuit to perform an action, the action of receiving sensor data from a mobile cleaning robot based on interaction between the mobile cleaning robot and the environment, and using the sensor data to generate boundaries of spaces that the mobile cleaning robot can pass through in the environment, the boundaries of which define at least partially impassable spaces in the environment, the impassable spaces including areas that extend beyond the boundaries, and using the impassable spaces and sensor data to generate modified boundaries of the environment.

[0124] In Example 20, the subject of Example 19 optionally includes the fact that the instruction causes the processing circuit to perform further actions to generate a map of the environment based on the modified boundaries.

[0125] In Example 21, the subject of Example 20 optionally includes the fact that the instruction causes the processing circuit to perform an operation to segment the map into rooms defined by the room boundaries, at least partially based on the modified boundaries.

[0126] In Example 22, the subject of Example 21 optionally includes a boundary-crossing space where the impassable space is observed by the mobile cleaning robot and is not traveled by the mobile cleaning robot.

[0127] Example 23 is a device that includes means for implementing any of Examples 1 through 20.

[0128] Example 24 is a system for implementing any of Examples 1 through 20.

[0129] Example 25 is a method for implementing any of Examples 1 through 20.

[0130] In Example 26, any one or any combination of the systems, apparatus, or methods described in Examples 1 through 25 may be configured as such that all of the described elements or options are available for use therein or for selection therefrom.

[0131] The above detailed description includes references to the accompanying drawings, which form part of the detailed description. The drawings illustrate, as examples, specific embodiments in which the present invention may be carried out. These embodiments are also referred to herein as “examples.” Such examples may include elements in addition to those illustrated or described. However, the inventors also intend examples in which only those illustrated or described elements are provided. Furthermore, the inventors also intend examples in which any combination or arrangement of those illustrated or described elements (or one or more embodiments thereof) is used in relation to a particular example (or one or more embodiments thereof) illustrated or described herein, or in relation to another example (or one or more embodiments thereof).

[0132] In the event of any conflicting usage between this Specified and any document incorporated herein, the usage herein shall prevail. In this Specified, the terms “including” and “in which” are used as plain English equivalents of the terms “comprising” and “wherein.” Furthermore, in the following claims, the terms “including” and “comprising” are open-ended; that is, any system, device, article, composition, formula, or process that includes elements in addition to those enumerated after such terms in a claim is still considered to be within the scope of that claim.

[0133] In this specification, the terms “a” or “an” are used to include one or more, independently of any other instances or uses of “at least one” or “one or more,” as is common in patent literature. In this specification, the term “or” is used to refer to non-exclusive “or,” such that “A or B” includes “A but not B,” “B but not A,” and “A and B.” In this specification, the terms “including” and “in which” are used as plain English equivalents of the terms “comprising” and “wherein.” Furthermore, in the following claims, the terms “including” and “comprising” are open-ended, meaning that any system, device, article, composition, formula, or process that includes elements in addition to those enumerated after such terms in a claim is still considered to be within the scope of that claim. Furthermore, in the following claims, terms such as “first,” “second,” and “third” are used merely as labels and do not impose any numerical requirements on those objects.

[0134] The above description is illustrative and not limiting. For example, the examples (or one or more embodiments thereof) described above may be used in combination with each other. Other embodiments may be used by those skilled in the art when considering the above description. The abstract is provided in accordance with Section 1.72(b) of the U.S. Patent Law Enforcement Rules (37 CFR §1.72(b)) to enable readers to quickly confirm the nature of the technical disclosure. The abstract is submitted with the understanding that it is not to be used to interpret or limit the scope or meaning of the claims. Also, in the forms for carrying out the invention described above, various features may be grouped together in order to simplify the disclosure. This should not be interpreted as meaning that any disclosed feature not claimed is essential to any claim. Rather, the subject matter of the invention may be fewer features than all of the particular disclosed embodiments. Accordingly, the following claims are incorporated into the forms for carrying out the invention described herein as examples or embodiments, and each claim stands on its own as a separate embodiment, and such embodiments are intended to be combined with each other in various combinations or arrangements. The scope of the present invention should be determined by reference to the appended claims, together with the entire scope of equivalents to which such claims are granted. [Explanation of Symbols]

[0135] 40 Environment Rooms 42, 42a-42e 44 beds 46 tables 48 Island 50, 50a~50e Floor surface 52. Rugs 54 Behavioral Control Zone 60 users 75 Debris 100 robots, mobile cleaning robots, cleaning robots, mobile robots 202 Main Unit 202a front 202b Rear 204 Cleaning assembly, cleaning head, extractor 205 Cleaning Roller 205a, 205b Cleaning rollers, rollers 208, 214, 244 motors Actuators 208a, 208b, 214a, 214b 210, 210a, 210b drive wheels 211 Caster Wheels 212 Controllers, Processors, Controllers 213 memory 218 Vacuum assemblies, vacuum systems 220 Airflow 224 Housing 234 Step sensor 238 Bumper 239, 239a, 239b Collision sensors 240 Image Capture Devices 241 Obstacle tracking sensor, sensor 242 Side brush, brush 243 Opening 245 Dirt sensor, battery 322 Cleaning bin 348 Intake duct 349 filters 400 Communication Networks 404 Mobile devices, mobile robots 405 User 406 Cloud Computing Systems 408 Autonomous robots, robots 432 processors 600 maps, unprocessed exclusive maps 636 Object, part, impassable part 636a~636n Object 700 maps 740 Boundary, boundary line 742 Room boundary, room boundary, boundary, boundary Room boundary lines: 742a~742n Rooms 744, 744a~744n 746 Opening 748 Modified boundary, boundary, single valid polygon or modified boundary 748a Exterior wall, modified boundary, boundary 748b Boundary 750a Passable space 752a Space that cannot be passed through 900 Maps 940 Boundary, boundary line 942a~942n Boundary Room boundary lines: 942a, 942b Room boundary lines: 942d, 942n Rooms 944, 944a~944n 950, 950a Passable space 952, 952a Impassable space 954, 954a, 954b, 954d Features, Objects Features of 954a~954n 956 poses 956a~956n Position or pose Room boundary, boundary 958, 958d Room boundaries for 958a, 958b, and 958n 1200 maps 1240 Boundary Line Rooms 1244a-1244d 1250, 1250a~1250d Passable space 1252 Negative space (or impassable space), impassable space 1252a~1252e Passable space 1258, 1258a~1258d Modified boundary lines 1400 Maps Rooms 1444, 1444a~1444n 1446 Opening 1452 Impassable objects and walls, impassable spaces, objects or impassable spaces 1452a~1452n Spaces or objects that cannot be passed through 1458 Modified boundary, wall, wall or border 1458a border, wall 1458a~1458n Border 1500 maps Rooms 1544 and 1544a 1550 Passable space 1558 Boundary Line 1560 Exterior Wall 1562 Interior wall 1700 Maps 1740 boundary 1742 Room boundary Room 1744 1748 Changed Boundaries 1750 Passable space 1752 Impassable space 1758 Changed Boundaries 1800 Machines, machines (for example, computer systems) 1802 Hardware Processor, Processor 1804 Main Memory 1806 Static memory (e.g., memory or storage for firmware, microcode, Basic Input / Output (BIOS), Unified Extensible Firmware Interface (UEFI), etc.), static memory 1808 Interlink (for example, bus) 1810 Display Unit 1812 Alphanumeric input device, input device 1814 User Interface (UI) Navigation Device, UI Navigation Device 1816 Large-capacity storage, memory devices (e.g., drive units) 1818 Signal Generating Devices 1820 Network Interface Device 1822 Machine-readable media 1824 Data structure or instruction, instruction 1826 Communication Network 1828 Sensor 1830 Output Controller

Claims

1. At least one non-temporary machine-readable medium which, when executed, includes an instruction causing a processing circuit to perform an operation, the operation being: The mobile cleaning robot receives sensor data based on the interaction between the mobile cleaning robot and the environment, Using the aforementioned sensor data, generate a boundary for a space that can be traversed by the mobile cleaning robot within the environment, wherein the boundary at least partially defines a space in the environment that cannot be traversed, and the non-traversable space includes an area that extends beyond the boundary. Using the impassable space and the sensor data, a modified boundary of the environment is generated. A non-temporary machine-readable medium for performing the following.

2. The aforementioned instruction, Based on the modified boundaries, generate a map of the environment. The at least one non-temporary machine-readable medium according to claim 1, which causes the processing circuit to further perform an operation to do the above.

3. The aforementioned instruction, Segmenting the map into rooms defined by room boundaries, at least partially based on the modified boundaries. The at least one non-temporary machine-readable medium according to claim 2, which causes the processing circuit to perform an operation to do so.

4. The map is segmented into rooms based at least partially on the impassable space, according to the at least one non-temporary machine-readable medium of claim 3.

5. The aforementioned instruction, Using the aforementioned sensor data, characterize the portion of the impassable space. The at least one non-temporary machine-readable medium according to claim 4, which causes the processing circuit to perform an operation to do so.

6. The aforementioned instruction, Modifying the room boundary based at least partially on the characterized portion of the impassable space. The at least one non-temporary machine-readable medium according to claim 5, which causes the processing circuit to further perform an operation to do so.

7. The portion of the at least one non-temporary machine-readable medium according to claim 6 is characterized using an image captured by the image capture device of the mobile cleaning robot.

8. The image is captured using a VSLAM process in at least one non-temporary machine-readable medium according to claim 7.

9. The aforementioned instruction, Using the aforementioned image, the height of the characterized portion is determined, Using the map and the height of the characterized portion, a three-dimensional representation is generated for each of the characterized portion. At least one non-temporary machine-readable medium according to claim 7 or 8, which causes the processing circuit to perform an operation to do so.

10. The aforementioned instruction, A three-dimensional map is generated based on the aforementioned map and based on the three-dimensional representation of the characterized portion. The at least one non-temporary machine-readable medium according to claim 9, which causes the processing circuit to perform an operation to do so.

11. At least one non-transient machine-readable medium according to any one of claims 1 to 10, wherein the impassable space includes a space beyond the boundary that is observed by the mobile cleaning robot and is not traveled by the mobile cleaning robot.

12. A method for generating an environmental map using a mobile cleaning robot, wherein the method is The steps include receiving sensor data based on the interaction between the mobile cleaning robot and the environment, A step of using the sensor data to generate a boundary of space that the mobile cleaning robot can pass through within the environment, wherein the boundary defines at least partially the space that the environment cannot pass through. The steps include generating a modified boundary of the environment using the impassable space and the sensor data, A step of generating a map of the environment based on the modified boundary. Methods that include...

13. The method according to claim 12, wherein the impassable space includes a space beyond the boundary that is observed by the mobile cleaning robot and is not traveled by the mobile cleaning robot.

14. The step of segmenting the map into rooms defined by room boundaries, based at least partially on the modified boundaries and the impassable spaces. The method according to claim 12 or 13, including the method described in claim 12 or 13.

15. The steps include: using the sensor data to characterize the portion of the impassable space; The steps of modifying the room boundary based at least partially on the characterized portion of the impassable space, The steps include determining the height of the characterized portion using the aforementioned image, A step of generating a three-dimensional representation for each of the characterized parts using the map and the height of the characterized parts, A step of generating a three-dimensional map based on the map and the three-dimensional representation of the characterized portion. The method according to claim 14, including the method described in claim 14.

Images