Software systems and method for a semi-autonomous floor cleaning apparatus for navigation of tight spaces

Abstract

Software, system and method of a semi-autonomous floor cleaning apparatus for navigation of tight spaces. A floor cleaning system of a semi-autonomous cleaning apparatus capable of autonomous movement and navigation of tight spaces for cleaning floors. The semi-autonomous cleaning apparatus consists of a cleaning system, a cleaning head assembly, a rear squeegee assembly, a water handling system and a plurality of sensors. The plurality of sensors including LiDAR sensors, cliff sensors, 3D cameras and data collection cameras are configured for data collection semi-autonomous navigation, floor cleaning and movement. Software for the apparatus may incorporate sensors and perception software modules, a machine learning module, an Edge following module, autonomy module, a safety system module, a Lidar filtering module, a semantic segmentation module, and an architecture to support New Enhanced Robot Foundation (NERF).

Description

SOFTWARE SYSTEMS AND METHOD FOR A SEMI-AUTONOMOUS FLOOR CLEANING APPARATUS FORNAVIGATION OF TIGHT SPACESCross Reference to Related Applications

[0001] The application claims priority to and the benefit of US Provisional Patent Application Serial No. 63 / 598927, entitled "SOFTWARE SYSTEMS AND METHOD FOR A SEMI-AUTONOMOUS FLOOR CLEANING APPARATUS FOR NAVIGATION OF TIGHT SPACES", filed on Nov. 14, 2023 and US Provisional Patent Application Serial No. 63 / 552457, entitled "SOFTWARE SYSTEMS AND METHOD FOR A SEMI- AUTONOMOUS FLOOR CLEANING APPARATUS FOR NAVIGATION OF TIGHT SPACES", filed on Feb. 12, 2024, and PCT Patent Application Serial No. CA2024051373, entitled "SYSTEM AND METHOD OF A SEMI- AUTONOMOUS FLOOR CLEANING APPARATUS FOR NAVIGATION OF TIGHT SPACES", filed on Oct. 18, 2024 the disclosures of which are incorporated herein by reference in their entirety.Background

[0002] The embodiments described herein relate to autonomous and semi-autonomous cleaning devices and more particularly, to a system and method for improved cleaning of indoor surfaces.

[0003] The use of autonomous and semi-autonomous cleaning devices configured to perform a set of tasks is known. For example, semi-autonomous devices or robots can be used to clean indoor surfaces such as department stores, shopping malls, airports and office buildings.

[0004] Most auto-scrubber cleaning systems or semi-autonomous cleaning devices are large and may lack intelligence. While they operate in a wide variety of environments and under a wide range of circumstances, they fail to adapt adequately to those changing circumstances in order to optimize and maximize cleaning performance, especially in smaller environments or tight spaces.

[0005] There is a desire to provide a semi-autonomous cleaning device with adjustable configurations to adequately adjust for tighter spaces.Summary

[0006] Software, system and method of a semi-autonomous floor cleaning apparatus for navigation of tight spaces. A floor cleaning system of a semi-autonomous cleaning apparatus capable of autonomous movement and navigation of tight spaces for cleaning floors. The semi-autonomous cleaning apparatus consists of a cleaning system, a cleaning head assembly, a rear squeegee assembly, a water handling system and a plurality of sensors. The plurality of sensors including LiDAR sensors, cliff sensors, 3D cameras and data collection cameras are configured for data collection, semi-autonomous navigation, floor cleaning and movement. Software for the apparatus may incorporate sensors and perception software modules, a machine learning module, an edge following module, autonomy module, a safety system module, a Lidar filtering module, a semantic segmentation module, an architecture to support New Enhanced Robot Foundation (NERF).Brief Description of the Drawings

[0007] FIG. 1 is a front perspective view of an exemplary semi-autonomous floor cleaning device.

[0008] FIG. 2 is a rear perspective view of the exemplary semi-autonomous floor cleaning device.

[0009] FIG. 3 is a front plan view of the exemplary semi-autonomous floor cleaning device.

[0010] FIG. 4 is a rear plan view of the exemplary semi-autonomous floor cleaning device.

[0011] FIG. 5 is a left-side view of the exemplary semi-autonomous floor cleaning device.

[0012] FIG. 6 is a right-side view of the exemplary semi-autonomous floor cleaning device.

[0013] FIG. 7 is a system diagram of the exemplary semi-autonomous floor cleaning device.

[0014] FIG. 8 is a block diagram of the robot interface of the exemplary semi-autonomous floor cleaning device.

[0015] FIGURES 9A to 9B are diagrams that provide information on sensors and the perception system.

[0016] FIG. 10 is a diagram illustrating an exemplary machine learning platform.

[0017] FIGURES 11A to 11B are diagrams illustrating information on edge following.

[0018] FIG. 12 is a block diagram illustrating the cleaning device autonomy software layout.

[0019] FIG. 13 is a system diagram illustrating the safety architecture of the exemplary cleaning device.

[0020] FIG. 14 is a system diagram illustrating components and information on the NERF system.

[0021] FIG. 15 is a system diagram of components of the NERF system.

[0022] FIG. 16 is a system diagram illustrating the NERF safety board firmware.

[0023] FIG. 17 is a system diagram illustrating the NERF concentrator board firmware.

[0024] FIG. 18 is a system diagram illustrating the NERF lighting board firmware.

[0025] FIG. 19 is a system diagram illustrating the NERF manual drive board firmware.

[0026] FIG. 20 is a system diagram illustrating the NERF scrubber board firmware.Detailed Description

[0023] An exemplary embodiment of an autonomous or semi-autonomous cleaning device is shown in Figures 1 - 6. FIG. 1 is a front perspective view of an exemplary semi-autonomous floor cleaning device. According to FIG. 1, exemplary semi-autonomous floor cleaning device 100 (otherwise known as cleaning device or cleaning robot) consists of illuminated logo 102 (e.g., Avidbots logo), eye lights 104 (i.e., lights that are shaped as eyes), cliff sensor pods 106, removal battery compartment 108, front Remote Monitoring (RM) camera 110, forward speaker 112, debris diverter module 114, headlights 116, curved body panel 118, status light 120, side access door 122, indicator lights 124, strobe lights 126 and 3D camera pylons 128.

[0024] FIG. 2 is a rear perspective view of the exemplary semi-autonomous floor cleaning device. According to FIG. 2, cleaning device 200 is shown with recovery tank lid 202, touchscreen display 204 having a touchscreen interface, manual drive handles 206 having a manual drive interface, on / off button 208, key switch 210, e-stop button 212, rear facing speaker 214, magnetic door latch 216, 3D camera housing 218 and integrated strobe light housing 220. 3D camera housing 218 further comprises a 3D camera, an integrated strobe light and LED indicator. Integrated strobe light housing 220 further comprises an integrated strobe and LED indicator.

[0025] FIG. 3 is a front plan view of the exemplary semi-autonomous floor cleaning device. FIG. 4 is a rear plan view of the exemplary semi-autonomous floor cleaning device. FIG. 5 is a left-side view of the exemplary semi-autonomous floor cleaning device. FIG. 6 is a right-side view of the exemplary semi- autonomous floor cleaning device.Specifications

[0026] The autonomous or semi-autonomous cleaning device delivers consistent commercial-grade cleaning to help keep a space healthy and safe, and features an attractive design. The cleaning device is enabled for use in, but not limited to, retail (stores & malls), health care institutions, education institutions, transportation (airports and transit) and warehouses (light duty).

[0027] Some features of the design include the following:• High productivity, advanced autonomy, and a user-friendly design• High performance and productivity• Tailored for close edge cleaning in tight spaces• Safe & Secure - designed to IEC 61508• Commercial-grade clean

[0028] Further specifications of an embodiment of the cleaning device include the following:• Dimensions: Tl" X 35" X 47" (width, length, height) or 692 mm X 893 mm X 1200mm• Voltage: 24 Volts• Noise Level: 63 dB(A)• Cleaning width: 22" (560 mm) using twin pad / brush• Close Edge Cleaning: 6" (150 mm)• Cleaning system option: Disc• Solution / Recovery capacity: 13 gallons / 48 liters• Continuous runtime: 3+ hours, option for exchangeable batteries• Downforce: 50 - 90 lb (23 - 40 kg)• Minimum U-turn Width: 51" (1.3 m)• Minimum Aisle Width: 35" (0.9 m)• Battery Technology: Lithium Iron Phosphate (LFP)System Overview

[0029] Fig. 7 is a system diagram of the exemplary cleaning device. According to FIG. 7, the cleaning device system 700 consists of the following components:Robot Interfaces

[0030] FIG. 8 is a block diagram of the robot interface of the exemplary semi-autonomous floor cleaning device. While the cleaning device is designed to be autonomous, it interacts with the environment, and individuals, in a number of ways. FIG. 8 shows the key interfaces 800 of the cleaning device and indicates the approximate manner in which the interactions take place.

[0031] According to FIG. 8, the cleaning device (i.e., cleaning robot 802) is connected to the following components or subsystems:• Horizontal surfaces (floors) 1204• Vertical surfaces (walls) 1206• Unclassified obstacles 1208• Environmental (thermal, humidity) 1210• Communication (i.e., RF, Wi-Fi, LTE) 1212• Control GUI (rear panel) 1214• Manual Control (steering / throttle) 1216• Character interface (front display) 1218• Emergency stop (button, guards) 1220• Maintenance (filling and cleaning) 1222Sensors & Perception

[0032] FIGURES 9A to 9C are diagrams that provide information on sensors and the perception system. FIG. 9A is a diagram that provides more information on odometry as it relates to sensors and perception.

[0033] FIG. 9A is a graph illustrating the heading estimate drift of encoders. According to graph 900 of FIG. 9A, the top line illustrates encoders and the bottom line illustrates a gyroscope.

[0034] According to FIG. 9A, info on odometry is as follows:

[0035] FIG. 9B is a diagram that provides more information on automatic recalibration as it relates to sensors and perception. According to FIG. 9B, calibration cleaning plan 910 is shown with arrows indicating the direction of travel.

[0036] According to FIG. 9B, info on automatic recalibration is as follows:

[0037] FIG. 9C is a diagram that provides more information on the perception system as it relates to sensors and perception.

[0038] According to further embodiments of the disclosure, sensors can be integrated into a perception system. The perception system consists of the following components:

[0039] FIG. 10 is a diagram illustrating an exemplary machine learning platform. According to FIG. 10, block diagram 1000 illustrates different components of the machine learning platform. The machine learning platform (MLP) 1002 consists of such components as cloud infrastructure 1010, data annotation 1012, data lake 1014, model registry 1016, experiment tracking 1018, model packing 1024, model deployment 1024 and performance monitoring 1022. According to FIG. 10B, the further communicates with an onboard machine learning platform (OMLP) 1004 consisting of a data collection 1026, anonymization 1028 and inference runner modules 1030. Furthermore, machine learning platform (MLP) 1002 will deploy models 1008 to the onboard marching platform (OMLP) 1004 and upload the data 1006 back to the MLP 1002.

[0040] According to the disclosure, the exemplary machine learning platform consists of the following components:Semantic Segmentation

[0041] According to the disclosure, semantic segmentation involves using machine learning (ML) and visual data to teach an algorithm (e.g., a deep neural network) to distinguish between ground and nonground spaces. The non-ground spaces are passed to the cleaning device path planning to avoid these areas. The algorithm may be programmed to detect low obstacles (e.g., cardboard boxes, low pallets, forklift tines) and other low-height obstacles. Semantic segmentation obstacle avoidance adds another layer of safety to perception safety protocols of a cleaning device. More information on semantic segmentation can be found in US patent application US18 / 481,367, entitled "SYSTEM AND METHOD OF SEMANTIC SEGMENTATION FOR A CLEANING DEVICE", the disclosure of which is incorporated herein by reference in its entirety.Edge Following

[0042] This subsystem of the cleaning device is intended to be an extension of the basic autonomy software system, specifically designed to achieve edge following (or wall following) in an environment. While this system also relies on input from multiple external sensors, and also relies on the safety zones defined by the safety system, this aspect of the robot is predominantly software focused, concentrating on the necessary control algorithms to plan and drive the robot as close as 15cm to a wall (or edge). Thissoftware resides on the Autonomy PC, and interfaces with the baseline Robot Software (which in turn interfaces with the cleaning device firmware).

[0043] FIGURES 11A to 11B are diagrams illustrating information on edge following. FIG. 11A is a block diagram of the various components of the modules associated with edge following. According to FIG. 11A, block diagram 1100 initiates with Mission Planner module 1102 starting a cleaning plan which is sent to the Coverage Planner module 1104. Coverage Planner module 1104 will then analyze the cleaning plan and create sectors at the Edge Following Planner module 1106.

[0044] According to FIG. 11A, the Edge Following Planner module 1106 will provide trajectory candidates to the Safety Zone Planner module 1108 which will return safety zones. Edge Following Planner module 1106 also creates edge tracking trajectory to the Motion Controller module 1110. Thereafter, motion controls are created and sent to the software (SW) / firmware (FW) bridge module 1112 that is implemented via the NERF framework.

[0045] According to FIG. 11A, the SW / FW Bridge module 1112 sends motion commands (or NERF protocols) to the Safety Controller module 1114. Thereafter, safe motion commands are then sent to the Motors module 1116.

[0046] According to FIG. 11A, block diagram 1100 further comprises the Encoders & IMU module 1120 sends encoder & IMU data stream to the Localization pipeline module 1122, as well as, encoder data to the Safety Controller module 1114. The Localization pipeline module 1122 sends pose and estimate data or feedback data to the Occupancy Map module 1124, as well as, to the Coverage Planner module 1104, the Edge Following Planner module 1106 and the Safety Zone module 1108.

[0047] According to FIG. 11A, block diagram 1100 further comprises Cameras and Lidars module 1126 which sends the camera and lidar data streams to the Sensor processing pipeline module 1128, and sends lidar data to the Safety Controller module 1114. Sensor processing pipeline module 1128 then sends sensor feed data to the Occupancy Map module 1124.

[0048] According to FIG. 11A, the final step is for the Occupancy Map module 1124 to create or update a map and feeds it to the Cover Planner module 1104 and Edge Following Planner module 1106 whereupon the process repeats.

[0049] According to FIG. 11A, the key software modules for edge following include Safety Zone Planner module 1108, edge following planner module 1106, Safety Controller module 1114 and the Occupancy Map 1124 module.

[0050] FIG. 11B is a diagram that illustrates edge following path generation overview 1150. According to FIG. 11B, the top diagram 1152 shows a holonomic path generation consisting of the following: a. Identification of control points at various wall distance inflations from local obstructions i. Red points 1165, orange points 1156, and yellow points 1158 ii. Lidar, 3D & no-go zone (NGZ) data b. Identification of control points along global reference path i. blue points 1160 ii. Identification & discounting of points within wall distance of local obstacles c. All data is in or transformed to the regional reference frame

[0051] According to FIG. 11B, the bottom diagram 1162 shows a non-holonomic path generation consisting of the following: a. Geometric connections defined between holonomic + global path control points to generate search graph (not all connections shown) b. Safety zone collision checks used to identify feasible and infeasible paths c. Best feasible path that achieves closest -to-target wall distance and furthest progression within the planning window is returned

[0052] According to the disclosure, edge following further comprises the following components and features:• Cleaning robot front differential drive (like a backwards tricycle drive), resulting in better stopping but also the characteristic rear body taper and more challenging path planning and tracking considerations• Safety zone footprints as function of speed & heading, considered in planning the EF path• Regional localization to provide jump-free accurate local pose estimation next to walls o Highly sensitive to accurate odometry - use high-speed scans + IMU + calibration• Edge following planning & controls - trajectory grooming replacement - stitch the edge following path with the coverage path at ~2 Hz - "always on" edge following o First, a holonomic edge following path is planned on a fine cost mapo Then this path is stitched with the coverage path based on whether the coverage path is in collision with the wall; if no, choose the coverage path, otherwise choose the EF path o Next, a kinematically and dynamically-feasible, safe curve path is fit to this path o Finally, the trajectory tracking controller (acting upon a virtual steering link, later transformed to differential commands in the Robot Software) tracks this curve accurately and speedily• Productivity and distance targets o 0.65 m / s target steady-state speed o 15 cm avg @ 2.5 cm std dev target distance, from edge of cleaning pads• Calibration of measurement systems (odometry, lidars, and cameras) needs to be very precise and accurate• 3-D cameras used in the cost-map along with lidars for footprint collision checksGeneral Autonomy

[0053] According to the disclosure, the cleaning device autonomy software stack typically refers to the three large autonomy-related software containers: planning, perception, and localization. These are integrated into the larger software stack which includes NERF, GUI, and ACC communication elements for example built on top of a multitasking operating system. Together these form the logic for the cleaning device, enabling intelligent decision-making in a changing environment. The figure below shows a representation of the autonomy subsystem, with the key containers identified in relation to other software containers in the Autonomy PC software stack.

[0054] FIG. 12 is a block diagram illustrating the cleaning device autonomy software layout. According to FIG. 12, the autonomy software is configured to perform the same tasks as earlier versions of the cleaning device in effectively the same manner, be backwards-compatible with the earlier cleaning device, and also satisfy the new autonomy requirements for this new version and future versions.

[0055] According to FIG. 12, block diagram 1200 illustrates the major containers and components of software in the autonomy stack. According to FIG. 12, local users 1202 will interact and operate cleaning device 1204 (or cleaning robot), as well as Command Center 1242. Remote users 1240 also interact with command center 1242.

[0056] According to FIG. 12, cleaning device 1204 further comprises of Actuators, Sensors, Indicators and Switches module 1206, NERF low-level platform module 1210 and Autonomy platform module 1220. NERF low-level platform module 1210 further comprises Safety Board & Safety Firmware module 1212, Cleaning Controller module 1214 and LED, Battery & Manual Drive Controllers module 1216.

[0057] According to FIG. 12, the Autonomy platform module 1220 comprises User Interface module 1222, Executive State Machine module 1224, Remote Services Adapters module 1226, Safety and Health Monitors module 1228, Localization and Perception module 1230, Planner and Controller module 1232, Diagnostics, Logging and Calibration module 1234 and Hardware Drivers module 1236. Hardware Drivers modules 1236 include such drivers as NERF, sensors, radios and other known drivers to operate cleaning device 1204.

[0058] According to FIG. 12, the Command Center module 1242 can be accessed and operated on the web or via a mobile device. Command Center module 1242 comprises Cleaning Plans and Reports module 1244, Robot Fleet Manager module 1246 and Account Management module 1248.Safety System

[0059] According to the disclosure, the safety system is essential to safe operation of the cleaning device. FIG. 13 is a system diagram illustrating the safety architecture of the exemplary cleaning device. According to FIG. 13, safety system 1300 consists of components that are safety related, safety relevant or have no safety impact. Failure of a safety-related (or safety-critical) component leads to a loss of safety function, whereas safety relevant components are closely tied to the safety function but do not directly impact it.

[0060] According to FIG. 13, safety system 1300 consists of safety board 1302 consisting of safety controller 1304 and motion controller 1306. Motion controller 1306 interacts bi-directionally with power button e-switch 1348. Safety controller 1304 also interacts with brakes 1308, drive motors 1310 and encoders 1312.

[0061] According to FIG. 13, battery management system (BMS) 1314 is connected to batteries 1316. BMS 1314 further interacts with motion controller 1306. Batteries 1316, safety controller 1304 and motion controller 1306 interacts with main contactor 1318, motor contactor 1322 and drive motor controller 1322.

[0062] According to FIG. 13, concentrator board 1324 comprises of non-safety computation element (NCSE) 1326, safety controller 1328 which receives input from time-of-flight (ToF) void sensor 1330. Safety controller 1328 of concentrator board 1324 also communicates with safety controller 1304 of safety board 1302.

[0063] According to FIG. 13, motion controller 1306 of safety board 1302 further communicates with manual controller 1332, lighting controller 1334, cleaning controller 1336 and hardware network manager 1338. Hardware network manager 1338 connects to autonomy controller 1340, then Lidar sensor 1342 which then loops back to safety controller 1304 of safety board 1302.

[0064] According to FIG. 13, keyswitch 1344 and Emergency stop 1346 also feed into safety controller 1304 of safety board 1302.Lidar Filtering

[0065] According to the disclosure, a further component of the cleaning device provides information on Lidar Filtering. Lidar Filtering is a first in first out (FIFO) system that utilizes Lidar sensors and filters for noise and dust. More information on noise and dust is as follows:• Noise (Unfiltered data is noisy - cobwebbing, sporadic random hits) o Simple spatial filtering addresses this o Bake spatial filtering into safety zone / perimeter guard algorithm o Can filter out objects smaller than 3-4 cm for safety of persons• Dust o May need temporal filtering if we can't afford to stop and wait (self recoverable)■ Increases safety zone length■ Does not appear to significantly increase SZ width due to FWD■ e.g. 3 measurement delay -> Still achieves 10.1 cm to wall @ 0.675 m / sNERF System

[0066] The New Enhanced Robot Foundation (NERF) is a subsystem of the cleaning device that incorporates all electronics and electrical interconnects necessary for communication and control of the mechatronic aspects of the robot in accordance with the New Enhanced Robot Foundation (NERF) requirements. This includes such items as Printed Circuit Board Assemblies (PCBAs), processors andcontrol units, computers and single-board-computers, as well as the associated firmware necessary for operation of those basic systems.

[0067] The specific NERF-compatible elements of cleaning device include:• Hardware Network Manager (HWNM)• Safety Board• Concentrator Board (for cliff detection)• Cleaning / Scrubber Controller• Manual Drive Controller• LED or Lighting Controllers

[0068] This subsystem does not include all electronic peripherals, such as motors or sensors which are connected to each of the respective subsystems. Communication between PC and NERF Boards occurs via Serial and / or CANbus communication.

[0069] FIG. 14 is a system diagram illustrating components and information on the NERF system. According to FIG. 14, NERF system 1400 consists of Autonomy PC module 1402, Network Manager Board module 1404, Upper Lighting Board module 1406, Bottom Lighting Board module 1408, Safety Board modules 1410 and 1412, Cleaning Controller Board module 1414 and Concentrator Board Module 1418.

[0070] According to FIG. 14, Autonomy PC module 1402 receives inputs from rear display 1420, RGB Cameras 1422, 3D Cameras 1424 and DCU Data Collection Unit (DCU) cameras 1426 via High-Definition Multimedia Interface (HDMI) 1428, Universal Serial Bus (USB) 1436 or ethernet port 1442 and 1492. DCU cameras 1426 is a set of cameras used to help inform current and future algorithm development particularly in the machine learning (ML) front. DCU cameras 1426 may be distinct from the front, rear, and ceiling RGB cameras, but have since been made into these same cameras. Furthermore, DCU cameras may be an expandable camera option.

[0071] According to FIG. 14, Autonomy PC module 1402 consists of a plurality of further components, including Radio Layer 1430, Graphical User Interface (GUI) module 1432, New Safety Categories module 1434, Autonomy module 1444, audio 1438 and audio driver 1440, Hailo-8 module 1446, robot model module 1448, simulation testing module 1450 and robot model incorporation module 1452. The Hailo-8 module 1446 is an onboard machine learning computer hardware, however, other similar hardware can also be implemented.

[0072] According to FIG. 14, Autonomy PC module 1402 further comprises the NERF Application Programming Interface (API) 1452, Subsystem Diagnostics module 1454, Subsystem Parameter Configuration or Calibration module 1458, Differential Drive Abstraction module 1456, Indicator Virtual Subsystem module 1460, Warning Light Virtual Subsystem module 1464, Brake Light Virtual Subsystem module 1466, Turn Signal Virtual Subsystem module 1468, Drive Virtual Subsystem module 1470, Safety Virtual Subsystem module 1472, Scrubber Virtual Subsystem module 1474, Lithium Battery Management module 1470, Power Management Virtual Subsystem module 1476, Network Manager Virtual Subsystem module 1478, Manual Controller Virtual Subsystem module 1480, Environmental Virtual Subsystem module 1482, Inertial Measurement Unit (IMU) Virtual Subsystem module 1484, NERF FOTA Subsystem module 1486.

[0073] According to FIG. 14, Autonomy PC module 1402 further comprises NERF bridge module 1462, NERF emulator module 1488, Differential Drive simulation module 1490. NERF bridge module 1462 includes serial interface and protocol serialization protocols and provides a serial interface to Network Manager Board module 1404.

[0074] According to FIG. 14, Autonomy PC module 1402 further communicates using protocol definitions 1494 and safety zone definitions 1496 with the different modules (i.e., Network Manager Board module 1404, Upper Lighting Board module 1406, Bottom Lighting Board module 1408, Safety Board modules 1410 and 1412, Cleaning Controller Board module 1414 and Concentrator Board Module 1418).

[0075] FIGURES 15 to 20 are further block diagrams describing further components of the NERF system. FIG. 15 is a system diagram illustrating the NERF network manager board firmware. According to FIG. 15, the network manager board module 1500 consists of the following components:• NERF Communications Interface 1502 via serial communication• Inertial Measurement Unit (IMU) Subsystem and NERF Messaging 1504• Network Manager Subsystem & NERF Messaging 1506• Network Manager FOTA & NERF Messaging 1508• IMU Data Interface (Serial Peripheral Interface (SPI)) 1510• Inertial Measurement Unit (IMU) 1512• Bootloader 1514 which supports serial & CANbus to serial conversion• Queue Management & Serial Packaging 1516• Serial Unpacking & CANbus routing 1516• NERF Communication Interface (CANOpen FDI) which communicates to NERF Bus and all otherNERF boards

[0076] FIG. 16 is a system diagram illustrating the NERF safety board firmware. According to FIG. 16, the safety board firmware 1600 consists of the following components:• NERF Communications Interface 1602 via serial communication• Power Subsystem & NERF Messaging 1604• Safety Subsystem & NERF Messaging 1606• Drive Subsystem & NERF Messaging 1608 including manual subsystem status read• Safety Board FOTA 1610• Bootloader 1612 which supports CANbus FD and SPI Bridge• Battery Management System (BMS) Monitoring 1614• Power Status and Control 1616• CANbus connection to BMS 1618• Discrete Inputs & Power Button 1620• Motion Controller 1622• CANOpen to Motor Drive 1624• Drive Coordinate Transformation 1626 (Wheel Diameter)• Wheel Velocity Limiter 1628• Diagnostic Blink Code Generator 1630• Discrete Outputs 1630 (including Power Button LED)• Non-safety computation element to safety computation element (NCSE-SCE) Communication (SPI) 1632

[0077] According to FIG. 16, safety board firmware 1600 further comprises the following external components including Battery Management System 1634, Battery 1636, Power Button 1638, Motor Drive 1640 and Power LED 1642.

[0078] FIG. 17 is a system diagram illustrating the NERF concentrator board firmware. According to FIG. 17, the concentrator board firmware 1700 consists of the following features:• Safety board (non-safe side) firmware 1702 and Safety board (SCE) firmware 1720• NERF Communications Interface (CANbus) 1704• Cliff Detection Subsystem & NERF Messaging 1706• Cliff Detection FOTA 1708• Bootloader 1710 which supports CANbus FD and SPI Bridge• NSCE-SCE Communications (SPI) 1712• SCE-NSCE Communications (SPI) 1722• Bootloader 1724 supporting SPI• Cliff Detection Logic 1726 including Cliff Detection Calibration parameters (i.e., LUT)• Cliff Detection Calibration Procedure Harder 1728• Cliff Detection Integrity Monitor 1730• Microprocessor Integrity Checks 1732• Fault Handler 1734• Sensor Filtering Post Processor 1736• Windowed Watchdog Provider 1738• Cliff Detection Sensor Manager 1740• Cliff Detection Sensor API (AFBR API Precompiled) 1742• Cliff Detection Sensor Driver 1744• Discrete Outputs 1746

[0079] FIG. 18 is a system diagram illustrating the NERF lighting board firmware. According to FIG. 18, the lighting board firmware 1800 connects to external Config Dip Switches 1802 and consists of the following internal components:• NERF Communications Interface (CANbus) 1804• Turn Signal Subsystem & NERF Messaging 1806• Warning Light Subsystem & NERF Messaging 1808• Indicator Light Subsystem & NERF Messaging 1810• Brake Light Subsystem & NERF Messaging 1812• Lighting Board FOTA Subsystem 1814• Bootloader 1816 which supports CANbus FD• Subsystem Launcher 1818• Discrete Inputs 1820 configuration• Light Hardware Interface 1822• Analog Outputs (Pulse-width modulation (PWM)) 1824

[0080] According to FIG. 18, config inputs set up certain subsystems and lights and are addressed by specific cards. Furthermore, the lighting board firmware 1800 consists of the following features:• Single FW supports all four possible Lighting Subsystems• Configuration DIP switches allow setup of each board for specific purpose within Switch• Switch Design includes 2 Lighting Boards o Warning + Turn Signals o Indicator + Braking

[0081] FIG. 19 is a system diagram illustrating the NERF manual drive board firmware. According to FIG.19, the manual drive board firmware 1900 consists of the following components:• NERF Communications Interface (CANbus) 1902• Manual Drive Board Subsystem & NERF Messaging 1904• Manual Drive Board FOTA & NERF Messaging 1906• Manual Command Generator 1908• Manual Drive CANbus Interface to Safety Board 1910• Bootloader 1912 which supports CANbus FD• Analog Inputs 1914 supporting average throttle and left / right throttle differential• Discrete Outputs 1916 supporting whether operator is present• Operator Presence Detection 1918

[0082] According to FIG. 19, the manual drive board firmware 1900 consists of the following features:• Receives Manual Control inputs via Analog Signals• Inputs are transformed into Left and Right wheel speed data, and are scaled by speed range value provided from PC via NERF• Left and Right manual speed commands delivered to Safety Board via NERF bus interface• Operator Present signal sent to Safety Board via discrete output - to be used to confirm operator is in manual control so that safety constraints can be removed temporarily

[0083] FIG. 20 is a system diagram illustrating the NERF scrubber board firmware. According to FIG. 20, the scrubber board firmware 2000 consists of the following components:• NERF Communications Interface (CANbus) 2002• Bootloader 2004 which supports CANbus FD• Scrubber Subsystem 2006• Scrubber Board FOTA SS 2008• Diagnostic Operation 2010• Fault Handler 2012• Cleaning Controller 2014• Heat Controller 2016• Pad Worn Detection 2018• Brush Controller 2020• Lift Controller 2022• Lift Actuator Controller 2024• Vacuum Controller 2026• Pump Controller 2028• Accessory Detection 2030• Overcurrent Open Circuit Protection 2032• Diagnostic Feedback 2034• Discrete Inputs 2036 supporting clean water empty and dirty water full• Average Inputs 2038 supporting dirty water level sensor, clean water level sensor and lift position sensor• Discrete Inputs 2040 supporting head downward limit software, head retracted position software, head extended position software• Business Motor Drive (SPI) 2042 supporting left brush motor command, right brush motor command, left revolutions per minute (RPM) feedback and other motor drive status• Pulse-width modulation (PWM) Voltage Command Outputs 2044 supporting cleaning head lift motor control, vacuum motor control and pump motor control• Discrete Outputs 2046 supporting water solenoid control• Average Inputs 2048 supporting battery voltage, accessory HALL sensor, left brush motor current, right brush motor current, vacuum motor current, cleaning head lift motor current, pump motor current

[0084] According to FIG. 20, the scrubber board firmware 2000 consists of the following features:• Cleaning Pressure Control• Water Flow Control o Pre-set Pressure levelso Modulation based on Speed o Pump compensation for battery level• Brush Speed Control• Accessory Detection o For Cleaning Pressure adjustment• Overcurrent Protection• Pad Wear Detection• Diagnostic Data• Manual / Diagnostic Operations

[0085] According to FIG. 20, the cleaning device (or robot) speed path curvature and battery voltage are used to modulate the speed pump.

[0086] According to the disclosure, a semi-autonomous cleaning apparatus for cleaning floor surfaces. The semi-autonomous cleaning apparatus comprises a frame, a computer processor, a plurality of sensors, a floor cleaning module and a computing system implementing a route planning system. The route planning system configured to take steps of receive a cleaning coverage path, calculate the cleaning coverage path to execute on the cleaning apparatus, plan an edge following path, plan a holonomic edge following path, stitch the edge following path with the cleaning coverage path to optimize the cleaning coverage path unless the cleaning coverage path is in collision with a wall, and favours the holonomic edge following path otherwise, fit a target curve path, based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones, to this stitched path.

[0087] According to the disclosure, the step of planning an edge following path of the cleaning apparatus further comprises planning a holonomic edge following path on a cost map. The step of fitting a target curve path to the stitched path further comprises fitting a target curve path to the stitched path based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones.

[0088] According to the disclosure, the apparatus further comprises a trajectory tracking controller, wherein the controller is enabled to cause the apparatus to track the target curve path. The plurality of sensors is selected from a list consisting of external sensors, Cliff sensors, dual LIDAR sensors, front camera, rear camera, 3D cameras and ceiling-facing cameras.

[0089] According to the disclosure, a method of route planning for cleaning floor surfaces of a semi- autonomous cleaning apparatus, the cleaning apparatus comprising a frame, a computer processor, a plurality of sensors, and a computing system. The method comprising the steps of receiving a cleaning coverage path, calculating the cleaning coverage path to execute on the cleaning apparatus, planning an edge following path, planning a holonomic edge following path, stitching the edge following path with the cleaning coverage path to optimize the cleaning coverage path unless the cleaning coverage path is in collision with a wall, and favours the holonomic edge following path otherwise, and fitting a target curve path, based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones, to this stitched path.

[0090] According to the disclosure, the step of planning an edge following path of the method further comprises planning a holonomic edge following path on a cost map. The step of fitting a target curve path to the stitched path of the method further comprises fitting a target curve path to the stitched path based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones.

[0091] According to the disclosure, the cleaning apparatus of the method further comprises a trajectory tracking controller wherein the controller is enabled to cause the apparatus to track the target curve path. The plurality of sensors of the cleaning apparatus of the method is selected from a list consisting of external sensors, Cliff sensors, dual LIDAR sensors, front camera, rear camera, 3D cameras and ceiling-facing cameras.

[0092] The functions described herein may be stored as one or more instructions on a processor- readable or computer-readable medium. The term "computer-readable medium" refers to any medium that can be accessed by a computer or processor. By way of example, and not limitation, such a medium may comprise RAM, ROM, EEPROM, flash memory, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can store program code in the form of instructions or data structures and that can be accessed by a computer. It should be noted that a computer-readable medium may be tangible and non-transitory. As used herein, the term "code" may refer to software, instructions, code or data that is / are executable by a computing device or processor. A "module" can be considered as a processor executing computer-readable code.

[0093] A processor as described herein can be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or anycombination thereof designed to perform the functions described herein. A general-purpose processor can be a microprocessor, but in the alternative, the processor can be a controller, or microcontroller, combinations of the same, or the like. A processor can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor may also include primarily analog components. For example, any of the signal processing algorithms described herein may be implemented in analog circuitry. In some embodiments, a processor can be a graphics processing unit (GPU). The parallel processing capabilities of GPUs can reduce the amount of time for training and using neural networks (and other machine learning models) compared to central processing units (CPUs). In some embodiments, a processor can be an ASIC including dedicated machine learning circuitry custom- built for model training and / or model inference.

[0094] The disclosed or illustrated tasks can be distributed across multiple processors or computing devices of a computer system, including computing devices that are geographically distributed.

[0095] The methods disclosed herein comprise one or more steps or actions for achieving the described method. The method steps and / or actions may be interchanged with one another without departing from the scope of the claims. In other words, unless a specific order of steps or actions is required for proper operation of the method that is being described, the order and / or use of specific steps and / or actions may be modified without departing from the scope of the claims.

[0096] As used herein, the term "plurality" denotes two or more. For example, a plurality of components indicates two or more components. The term "determining" encompasses a wide variety of actions and can include calculating, computing, processing, deriving, investigating, looking up (e.g., looking up in a table, a database or another data structure), ascertaining and the like. Also, "determining" can include receiving (e.g., receiving information), accessing (e.g., accessing data in a memory) and the like. Also, "determining" can include resolving, selecting, choosing, establishing and the like.

[0097] The phrase "based on" does not mean "based only on," unless expressly specified otherwise. In other words, the phrase "based on" describes both "based only on" and "based at least on."

[0098] While the foregoing written description of the system enables one of ordinary skill to make and use what is considered presently to be the best mode thereof, those of ordinary skill will understand andappreciate the existence of variations, combinations, and equivalents of the specific embodiment, method, and examples herein. The system should therefore not be limited by the above described embodiment, method, and examples, but by all embodiments and methods within the scope and spirit of the system. Thus, the present disclosure is not intended to be limited to the implementations shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

ClaimsWhat is claimed:

1. A semi-autonomous cleaning apparatus for cleaning floor surfaces comprising: a frame; a computer processor; a plurality of sensors; a floor cleaning module; and a computing system implementing a route planning system, the route planning system configured to take the following steps: receive a cleaning coverage path; calculate the cleaning coverage path to execute on the cleaning apparatus; plan an edge following path; plan a holonomic edge following path; stitch the edge following path with the cleaning coverage path to optimize the cleaning coverage path unless the cleaning coverage path is in collision with a wall, and favours the holonomic edge following path otherwise; fit a target curve path, based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones, to this stitched path.

2. The cleaning apparatus of claim 1, wherein the step of planning an edge following path further comprises planning a holonomic edge following path on a cost map.

3. The cleaning apparatus of claim 1, wherein the step of fitting a target curve path to the stitched path further comprises fitting a target curve path to the stitched path based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones.

4. The cleaning apparatus of claim 1, wherein the apparatus further comprises a trajectory tracking controller, wherein the controller is enabled to cause the apparatus to track the target curve path.

5. The cleaning apparatus of Claim 1 wherein the plurality of sensors is selected from a list consisting of external sensors, Cliff sensors, dual LIDAR sensors, front camera, rear camera, 3D cameras and ceiling-facing cameras.

6. A method of route planning for cleaning floor surfaces of a semi-autonomous cleaning apparatus, the cleaning apparatus comprising a frame, a computer processor, a plurality of sensors, and a computing system, the method comprising the steps of: receiving a cleaning coverage path; calculating the cleaning coverage path to execute on the cleaning apparatus; planning an edge following path; planning a holonomic edge following path; stitching the edge following path with the cleaning coverage path to optimize the cleaning coverage path unless the cleaning coverage path is in collision with a wall, and favours the holonomic edge following path otherwise; fitting a target curve path, based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones, to this stitched path.

7. The method of claim 6, wherein the step of planning an edge following path further comprises planning a holonomic edge following path on a cost map.

8. The method of claim 6, wherein the step of fitting a target curve path to the stitched path further comprises fitting a target curve path to the stitched path based on kinematically and dynamically feasible curves and dynamic, velocity-dependent safety zones.

9. The method of claim 6, wherein the apparatus further comprises a trajectory tracking controller wherein the controller is enabled to cause the apparatus to track the target curve path.

10. The method of claim 6 wherein the plurality of sensors is selected from a list consisting of external sensors, Cliff sensors, dual LIDAR sensors, front camera, rear camera, 3D cameras and ceiling-facing cameras.T1

Images