Toilet bowl cleaning system

Abstract

According to a first aspect of the present invention, there is provided a toilet bowl cleaning system 5 having: a robotic arm; a base to which the robotic arm is coupled, the base comprising a plurality of cleaning head attachments for the robotic arm; a camera assembly; and a computing platform configured to analyse received images from the camera assembly for presence of a toilet bowl with an algorithm trained to differentiate between parts of a toilet bowl and detect the toilet bowl part on which a stain is located; select, based on the stain location, a cleaning head attachment for use to clean the stain from the plurality of cleaning head attachments; and transmit a command to the robotic arm to couple with the selected cleaning head attachment for cleaning the stain.

Description

[0001] Title of invention: Toilet bowl cleaning system

[0002] FIELD

[0003]

[0001] The present invention relates to a toilet bowl cleaning system with Al capability.

[0004] BACKGROUND

[0005]

[0002] Existing autonomous toilet bowl cleaning robots focus on being able to clean the interior and exterior surfaces of the toilet bowl. However, they do not consider other operational factors, such as effectiveness of cleaning, speed of cleaning and prevention of cleaning tool contamination. Several of these robots only perform routine cleaning sequences and are not able to spot clean (target cleaning action on detected areas with stains). Another shortcoming is that existing robots require a long time to calibrate to a new environment for deployment, making it difficult to scale.

[0006]

[0003] The present invention seeks to address the above shortcomings.

[0007] SUMMARY OF THE INVENTION

[0008]

[0004] According to a first aspect of the present invention, there is provided a toilet bowl cleaning system having: a robotic arm; a base to which the robotic arm is coupled, the base comprising a plurality of cleaning head attachments for the robotic arm; a camera assembly; and a computing platform configured to analyse received images from the camera assembly for presence of a toilet bowl with an algorithm trained to differentiate between parts of a toilet bowl and detect the toilet bowl part on which a stain is located; select, based on the stain location, a cleaning head attachment for use to clean the stain from the plurality of cleaning head attachments; and transmit a command to the robotic arm to couple with the selected cleaning head attachment for cleaning the stain.

[0009] BRIEF DESCRIPTION OF THE DRAWINGS

[0010]

[0005] Representative embodiments of the present invention are herein described, by way of example only, with reference to the accompanying drawings, wherein:

[0011]

[0006] Figure 1 shows a perspective view of a toilet bowl cleaning system in accordance with one embodiment of the present invention.

[0012]

[0007] Figure 2 shows an enlarged view of a distal end of the robotic arm of the toilet bowl cleaning system of Figure 1.

[0013]

[0008] Figure 3 shows a marker arrangement used by the toilet bowl cleaning system.

[0009] Figure 4 is a block diagram for architecture of a software platform used to manage one or more toilet bowl cleaning systems.

[0014]

[0010] Figure 5 shows interaction of various modules when performing a cleaning operation.

[0015]

[0011] Figure 6 shows a GUI used to facilitate creation, editing, planning and analysis of cleaning motions executed by the robotic arm.

[0016]

[0012] Figure 7 shows a process flow used by a developed perception pipeline builder module for stain detection.

[0017]

[0013] Figure 8 shows high-fidelity synthetic images used to train the toilet bowl cleaning system.

[0018]

[0014] Figure 9 shows several internal components of the base of the toilet bowl cleaning system.

[0019]

[0015] Figure 10 is a perspective view of storage for cleaning head attachments used by the toilet bowl cleaning system.

[0020]

[0016] Figure 11 shows communication between a robot software platform used for, amongst other purposes, fleet management and an Al platform used by the robot software platform.

[0021] DETAILED DESCRIPTION

[0022]

[0017] The present application relates to an autonomous system designed to clean a toilet bowl. While cleaning the toilet bowl, the cleaning system may also clean its surroundings, such as the cubicle area in which the toilet bowl is located. The cleaning system includes the following components, such as a base, a robotic arm, a plurality of cleaning head attachments, a camera assembly and a computing platform. The base is mechanised from having movement means such as wheels and a motor mechanism to allow the base to move to execute different cleaning tasks (i.e. to clean different toilet bowls in the same or different toilets); and thus provides a mount for components that perform physical cleaning such as the robotic arm and the plurality of cleaning head attachments.

[0023]

[0018] The cleaning task is performed by a cleaning head attached to the robotic arm, whereby the actuation of the robotic arm during the cleaning task is controlled by instructions received from the computing platform, which confers autonomy to the cleaning system. In an integrated implementation, the computing platform is housed within the base and the camera assembly which feeds images used to facilitate the cleaning task is mounted on the distal end of the robotic arm. However, in a segregated implementation, elements of the computing platform may be located remotely from the base (e.g. hosted in a separate server), such as controls relating to fleet management and robotic arm movement.

[0024]

[0019] In more detail, the computing platform is configured to analyse received images from the camera assembly for presence of a toilet bowl with an algorithm trained to differentiate between parts of a toilet bowl and detect the toilet bowl part on which a stain is located. Example of algorithms include perception based machine learning models that can be trained to learn, identify and categorise different objects from a large dataset of labelled images. Prior to deployment, the algorithm is fed images of clean toilet bowls and dirty toilet bowls having stains at different parts, with labels for each of the

[0025]

[0026] different objects in the images (such as one or more of a pan, seat, seat cover, tank, cubicle wall, toilet floor, stain) for the algorithm to learn to distinguish between toilet bowl parts through feature extraction. The training also results in the trained algorithm being able to identify whether a segment on any of these parts has a stain. A stain refers to presence of foreign matter, such as urine or faeces splatter, or remanent toilet paper. Accordingly, in this application, a toilet refers to an area having one or more cubicles where each has a toilet bowl. Toilet bowl refers to a sanitary hardware unit to collect human waste, the hardware unit having one or more the above mentioned parts of the pan to collect the human waste, the seat and seat cover (both of which are typically hinged and bolted to the pan) and the tank (also known as “cistern”) to store water to flush the pan.

[0027]

[0020] The computing platform is also configured with an algorithm that selects, based on the stain location, a cleaning head attachment for use to clean the stain from the plurality of cleaning head attachments and transmits a command to the robotic arm to couple with the selected cleaning head attachment for cleaning the stain. A cleaning head attachment refers to any cleaning tool such as a brush with bristles, a sponge, a microfiber brush, a mop, a gripper and a cloth.

[0028]

[0021] In one implementation, the analysis for toilet bowl stains and subsequent deployment of the selected cleaning head attachment to clean the stains is part of a spot cleaning operation performed after the toilet bowl cleaning system first performs a generic cleaning operation, such as a simple flushing of the toilet bowl pan; or spraying detergent across the toilet bowl followed by a water rinse. This spot cleaning operation is a contact based approach, advantageous for countering tough stains not removed during the contactless approach of the detergent and water rinse spray.

[0029]

[0022] Information about the toilet bowl configuration (such as number of toilet bowl parts; measurements for each toilet bowl part; and assembly arrangement of the toilet bowl parts) may be stored into one or more libraries of a software platform with a planning tool that uses a motion design module. The motion design module provide means to use the information, for example, the dimensions and design of the toilet bowl part to design the cleaning motion of the robotic arm to clean the various parts of the toilet bowl. Designed cleaning motions for different toilet bowls are stored into one or more libraries of the software platform with their respective unique object IDs.

[0030]

[0023] A marker arrangement, which received images from the camera assembly are analysed for presence thereof, facilitates retrieval of the object IDs as follows. In one approach, the marker arrangement has two parts during calibration, a permanent part located on a fixture in a vicinity of the toilet bowl; and a temporary part on the toilet bowl to locate the toilet bowl relative to the permanent part. The temporary part is removed after calibration as it is not required during the cleaning operation and to not obstruct the cleaning operation. Detection of the permanent part in the received images from the camera assembly during a cleaning operation will cause the computing platform to be aware of the location of the toilet bowl by factoring the calibration from the marker arrangement and provide instructions to the toilet bowl cleaning system to position itself to the toilet bowl to commence cleaning. The advantage of the calibration from the marker arrangement is that co-ordinates of the toilet bowl are

[0031]

[0032] calculated from preset data which allows the toilet bowl cleaning system, including the robotic arm, to position itself more quickly to a starting pose to clean the toilet bowl, compared to having to derive the location of the toilet bowl from received images of a toilet bowl.

[0033]

[0024] In addition, the object IDs referring to the cleaning motions designed for the toilet bowl may be coded into the fixed marker / permanent part for quick retrieval. Detection of the permanent part in the received images from the camera assembly during a cleaning operation will also cause the computing platform to be aware of a shortlist of cleaning motions that can be selected to clean the various toilet bowl parts.

[0034]

[0025] After the robotic arm has coupled with the selected cleaning head attachment, a cleaning motion from the shortlist of cleaning motions for the robotic arm will be selected to clean one or more areas with stain. Each cleaning motion may be constructed from combining pre-programmed motions to clean one or more parts of the toilet bowl. The cleaning motion forms part of a path that starts after the robotic arm is coupled with the selected cleaning head attachment, followed by guiding of the robotic arm to the location of the detected stain, a movement sequence used by the robotic arm to remove the stain, and ending with the robotic arm being returned to rest. The software platform planning tool may reference one or more libraries for designing the cleaning motion for the robotic arm based on any one or more of: conceivable stain location, the toilet bowl configuration and surroundings of the toilet bowl; and store the designed cleaning motion with unique Object IDs for quick referencing. Examples of libraries include: i) a toilet bowl environment which provides information on the layout of the surroundings of the toilet bowl, such as fixtures or obstacles around the toilet bowl for the robotic arm to avoid when being guided to the location of the detected stain or when cleaning the stain; ii) cleaning skills which result in one or more of the following: having the robotic arm flush the toilet bowl using a retractable prod; causing an appropriate nozzle on the robotic arm to emit one or more of detergent, water, disinfectant; or having the robotic arm pick up a brush and execute a scrubbing motion or a motion that wipes dry a damp / wet toilet bowl part; and iii) cleaning work packages which provides information on the parts and sequence to clean for a toilet bowl which can differ for different end users. The referencing of such libraries enhances the process to design cleaning motions to achieve cleaning efficiency and efficacy.

[0035]

[0026] Summarising, the marker arrangement serves two purposes: deriving the location of the toilet bowl; and referencing of cleaning motion libraries. It will be appreciated that one works independent of the other, i.e. the marker arrangement code responsible for toilet bowl location is separate from the marker arrangement code that facilitates referencing of cleaning motion libraries.

[0036]

[0027] Various embodiments of the toilet bowl cleaning system in accordance with the present invention are now described with reference to the drawings, where like reference characters generally refer to the same features across the drawings. It will be appreciated that the tool storage system may have more components that are not described for the sake of simplicity.

[0037]

[0038]

[0028] Figure 1 shows a perspective view of a toilet bowl cleaning system 100 in accordance with one embodiment of the present invention. In use, a base 102 of the toilet bowl cleaning system 100 navigates autonomously into a toilet bowl cubicle 130 and cleans the toilet bowl without human interventions. The toilet bowl cleaning system 100 is capable of performing routine cleaning, which includes using and changing various cleaning tools, to effectively clean the entire exterior and interior surfaces of the toilet bowl: as well as perform spot cleaning, which includes being able to detect the presence of stains and in real-time, select, construct and execute a cleaning action to remove the stain.

[0039]

[0029] One end of a robotic arm 104 is coupled to a base 102 to be pivotable about the mounting point. The toilet bowl cleaning system 100 may have more than one robotic arm, although only one is shown. Each of the one or more robotic arms 104 has a layout that may have a plurality of members, each coupled to another member by a respective joint 106. Each joint 106 allows the coupled member to have one or more of the following motions: yaw, pitch and roll. The interconnection provides each robotic arm 104 with several degrees of freedom, allowing movement along and / or about multiple axes, so that the robotic arm 104 exhibits anthropomorphic articulation.

[0040]

[0030] The base 102 has storage 118 for a plurality of cleaning head attachments 110 for the robotic arm 104. The cleaning head attachments 110 include different sizes of one or more of the following: a brush with radially protruding bristles, a flat brush with downward extending bristles, a microfibre brush, a sponge, a mop, a gripper and a cloth.

[0041]

[0031] One or more of the cleaning head attachments 110 is motorised, drawing power when coupled with the robotic arm 104 and / or when stored.

[0042]

[0032] During use, the robotic arm 104 will actuate to dock with a selected cleaning head attachment 110. The cleaning head attachment 110 mounts at a distal end of the robotic arm 104, seen more clearly in the enlarged view of Figure 2. The distal end of the robotic arm 104 has a flange 240 to which the cleaning head attachment 110 couples via a mating connector 242 (either of female or male configuration). The mating connector 242 provides an end effector that allows the robotic arm 104 to autonomously pick a cleaning head attachment 110 from a stowed storage position, secure and lock the cleaning head attachment I 10 onto the robot arm end effector. After use, the mating connector 242 unlocks the cleaning head attachment 110 during stow back.

[0043]

[0033] Other peripheral components may be connected to the flange 240, such as a camera assembly 108. The camera assembly 108 may have any one or more of the following non-exhaustive components: a camera to record and transmit a video feed to facilitate computer vision related tasks, the camera having digital or optical zoom capability; a lighting array (activated during low light conditions to enhance the video feed); and controls for the lighting array.

[0044]

[0034] Mounting the camera assembly 108 proximate to the cleaning head attachment 110 when coupled to the robotic arm 104 allows the camera assembly 108 to capture a clear field of view of the toilet bowl parts, by moving the robotic arm 104 to appropriate positions. The clear field of view' image capture is required for the inspection task to check for stains during a cleaning operation. Alternatively,

[0045]

[0046] the camera assembly may be mounted directly to the base. In another embodiment (not shown), another camera assembly may be mounted directly to the base 102 and positioned to enhance the localisation and movement of the base 102 (e.g. when moving between toilet bowls, or aligning the toilet bowl cleaning system 100 to toilet bowl).

[0047]

[0035] The toilet bowl cleaning system 100 has a computing platform 116 (not visible from Figure 1, but location indicated) which is secured onto the base 102. The computing platform 116 controls the operation of the toilet bowl cleaning system 100, such as providing instructions to move the robotic arm 104 and instructions to move the base 102. The base 102 has a router I 12 to communicate I 14 with a robot software platform 400 (described in further detail below with reference to Figures 4 and 11).

[0048]

[0049]

[0036] The computing platform 116 depends on images received from the camera assembly 108 to perform a cleaning operation after the base 102 reaches a toilet bowl to be cleaned. The computing platform 116 harnesses artificial intelligence, in that the received images are processed for presence of a toilet bowl with an algorithm trained to differentiate between parts of a toilet bowl and detect the toilet bowl part on which a stain is located. As mentioned above, the algorithm may be a perception based machine learning model trained to identify a toilet bowl from received images and whether stains are present and on which part, prior to deployment. The computing platform 116 selects, based on the stain location, a cleaning head attachment 110 for use to clean the stain from the plurality of cleaning head attachments. Other factors that the computing platform 116 may consider when selecting a cleaning head attachment 110 include the size of the stain and the nature of the stain (such as whether it is due to urine, faeces or toilet paper). The computing platform 116 then transmits a command to the robotic arm 104 to couple with the selected cleaning head attachment 110 for cleaning the stain.

[0050]

[0037] A cleaning operation may follow the following steps. The robotic arm 104 actuates to the storage I 18 for the mating connector 242 to dock with the selected cleaning head attachment I 10. The robotic arm 104 with the cleaning head attachment 110 is then moved to the toilet bowl part with the stain to execute a cleaning motion to remove the stain. If necessary, the base 102 may also adjust to better position the robotic arm before the robotic arm 104 is moved to the stained toilet bowl part. The robotic arm 104 may then move to clean other stains on the same toilet part with the same cleaning head attachment 110 if it is detected that there is more than one stain (e.g. faeces splatter). When cleaning other stained toilet bowl parts, the robotic arm 104 may return to the storage 118 to swap a cleaning head attachment 110 if the present cleaning head attachment 110 is unsuitable to clean these other stains for reasons such as the nature of the stain; or the dimension or shape of the toilet bowl part on which the stain is located. For example, the robotic arm 104 may swap from a cleaning head attachment 110 used for cleaning the interior surface of the toilet bowl to a different cleaning head attachment 110 used for cleaning the exterior surface of the toilet bowl. After confirming (through the camera assembly 108 video feed) that the stain is removed, the robotic arm 104 may return to the storage 118 to swap for a cleaning head attachment 110 that can dry the cleaned toilet bowl parts (such as a cloth). The computing platform 116 may also verify for stain removal from images received from the camera assembly 108

[0051]

[0052] after the robotic arm 104 has cleaned the stain. In response to detecting that the stain remains, a command may be transmitted to re -clean the stained toilet part or request for human intervention. The requested human intervention may include getting the stain manually cleaned by sending a notification using router I 12 for a cleaner to follow up, while the command for the request for human intervention may already be sent after a first attempt by the robotic arm 104 to clean the stain or after a second or subsequent failed attempts. At the end of the cleaning operation, the robotic arm 104 may then actuate to storage 118 which acts also as a waste water tank (described in greater detail with Figure 10) for the cleaning head attachment I 10 to undergo debris removal and disinfection after stowing the cleaning head attachment 110.

[0053]

[0054]

[0038] As mentioned above, the analysis for toilet bowl stains and subsequent deployment of a selected cleaning head attachment to clean the stains is part of a spot cleaning operation. Before the spot cleaning operation, the toilet bowl cleaning system 100 may perform generic cleaning of interior and exterior surfaces of the toilet bowl through one or more of the following steps: moving the robotic arm 104 so that a retractable prod (not shown) disposed on the robotic arm 104 can extend to actuate the toilet bowl flush to flush the toilet bowl pan; and positioning the robotic arm 104 so that a cleaning nozzle 244 can spray detergent across the toilet bowl. One or both of the flushing operation and the cleaning nozzle 244 operation may be performed in any sequence and repeated if necessary. As shown in Figure 2, an outlet end of the cleaning nozzle 244 is proximate to the mating connector 242 (i.e. where the cleaning head attachment 110 couples). The performance of generic cleaning can also include moving the robotic arm 104 to couple with a cleaning head attachment 110, thereafter move the robotic arm 104 with the cleaning head attachment 110 to a toilet bowl part and execute a cleaning motion to clean the toilet bowl part. As mentioned, this generic cleaning may be performed regardless of whether a stain is detected and may be done before the camera assembly 108 detects for stains. When the robotic arm 104 is used during the generic cleaning without a cleaning head attachment 110, it is contactless in that a gap exists between the cleaning nozzle 244 and the surfaces sprayed w'ith the detergent.

[0055]

[0039] Returning to the spot cleaning operation, the computing platform 116 is further configured to analyse the received images from the camera assembly 108 for artefacts which improve cleaning efficiency before commencing a cleaning operation. Figure 3 shows images 302 (pre calibration) and 304 (post calibration) of such an artefact, being a marker arrangement 308 that uses fiducial marker based deployment to which the camera assembly 108 analyses for presence thereof and provides the computing platform 116 with the data extracted from the marker arrangement 308.

[0056]

[0040] Fiducial marker-based deployment shortens the deployment lead time of the toilet bowl cleaning system 100. In the present embodiment, the method employs a two-marker approach for toilet bowl localisation. A temporary calibration marker 306A is placed on the toilet bowl pan 310 and a permanent localisation marker 306B on a wall are captured together in a single image 302. Post calibration (refer image 304), softw'are analysis of an image of the marker 306B taken by the camera assembly 108 before the robotic arm 104 with a selected cleaning head attachment 110 moves tow'ards

[0057]

[0058] the toilet bowl allows the toilet bowl cleaning system 100 to reference stored data to accurately determine the position of the toilet bowl and pre-programmed cleaning motions required for cleaning the toilet bowl. Accordingly, the marker arrangement 304 comprises two parts during cahbration: a permanent part 306B located on a fixture 301 in a vicinity of the toilet bowl; and a temporary part 306 A on the toilet bowl to locate the toilet bowl relative to the permanent part 306B, the temporary part 306A being removed after calibration. The fiducial marker-based deployment is advantageous for enabling fast transferring of data on the toilet bowl location and pre-planned cleaning motion to the toilet bowl cleaning system 100 during operation (such as to clean multiple toilet bowls in different toilets).

[0059]

[0041] As the computing platform 116 obtains data on the toilet bowl location and pre-programmed cleaning motion from the marker 306B when moving the toilet bowl cleaning system 100 and robotic arm 104 to clean any toilet bowl part, the cleaning operation is done faster. This is because the data extracted from the marker 306B is used, for example, to retrieve information about the object IDs for pre-programmed cleaning motion for the toilet bowl, so that movement instructions provided by the computing platform 116 to the robotic arm 104 are from pre-programmed cleaning motions retrieved from computing platform 116 using the object IDs.

[0060]

[0042] The cleaning motion of the robotic arm 104 is pre-programmed, using a robot software platform 400 shown in Figures 4 and 11. The robot software platform 400 has a deployment platform module 402; a robot arm module 404 having a planning tool (having a motion design module 410) and libraries 412; and mobile robot base module 406 having a navigation stack 428 and simulation module 430.

[0061]

[0043] The deployment platform module 402 is used to dispatch the base 102 of a toilet bowl cleaning system 100 automatically based on toilet status, cleaning schedule or via user commands on the deployment platform module 402 of the robot software platform 400 or through a middleware platform. The deployment platform module 402 also provides means to manage and monitor a fleet of toilet bowl cleaning systems 100 via communication channel 1102 (see Figure 11), which includes receiving and logging alerts, assigning each toilet bowl cleaning system 100 to a different group of one or more toilets; and collect data from the toilet bowl cleaning systems 100 for future planning purposes.

[0062]

[0044] The robot arm software module 404 provides for the planning of cleaning motions using the motion design module 410, where the designed cleaning motions can be used to update the computing platform 116 of the toilet bowl cleaning system 100 with the latest cleaning motions. The motion design module 410 can reference one or more libraries 412 to program the cleaning motion of the robotic arm 104.

[0063]

[0045] Examples of the libraries 412 include: i) a toilet bowl environment 414 which provides information on the layout of the surroundings of the toilet bowl, such as fixtures or obstacles around the toilet bowl for the robotic arm 104 to avoid when being guided to the location of the detected stain or when cleaning the stain; ii) cleaning work packages 416 which provides information on the parts and sequence to clean for a toilet bowl which can differ for different end users; and iii) cleaning skills 418 which result in one or more of the following: having the robotic arm 104 flush the toilet bowl by

[0064]

[0065] depressing the flush button; causing an appropriate nozzle 244 on the robotic arm 104 to emit one or more of detergent, water, disinfectant; or having the robotic arm 104 execute a scrubbing motion or a motion that wipes dry a damp / wet toilet bowl part. The referencing of such libraries 412 allows the design of suitable cleaning motions for the robotic arm 104 to, for example, avoid fixtures / obstacles in the toilet environment to perform a cleaning task.

[0066]

[0046] The robot arm module 404 provides one or more cleaning motions that the robotic arm 104 may be commanded to execute. One method is to transfer the packages or updates of cleaning motions from robot arm module 404 to the computing platform I 16 is through the communication I 14 channel (refer Figure 1; also see communication channel 1102 in Figure 11).

[0067]

[0047] Figure 5 illustrates how the computing platform 116 performs control of the operation of the toilet bowl cleaning system 100. A cleaning task management and execution module 502 receives the outputs from the external camera module 508, the calibration module 510 and is pre-loaded with designed cleaning motions from the cleaning motion design module 410,to decide how' the toilet bowl cleaning system 100 should execute a cleaning operation to clean a detected stain. Accordingly, the computing platform 116 references one or more libraries for selection of a cleaning motion for the robotic arm 104 based on any one or more of: the stain location, the toilet bowl configuration and surroundings of the toilet bowl. The computing platform 116 then transmits a command for the robotic arm 104 to execute the selected cleaning motion. This command involves the cleaning task management and execution module 502 sending a command signal to a robot arm controller module 506. If the base 102 is required to be moved for the robotic arm 104 to execute the cleaning operation, such as to an assigned toilet; to a cubicle in a toilet; or minor adjustments around the toilet bowl which is being cleaned, the cleaning task management and execution module 502 sends an appropriate command to a mobile base controller module 504 to have the base 102 execute the necessary movement.

[0068]

[0048] Returning to Figure 4, the motion design module 410 provides a customisable GUI 602 to facilitate the creation, editing, design and analysis of cleaning motions for the robotic arm 104 within a reconstructed working environment, as shown in Figure 6. A perception pipeline builder module 426 is used to train perception models, including a stain detection model trained to detect stains on the toilet bowl and know which part of a toilet bowl the stain resides on. This perception capability when deployed to computing platform 116 enables the toilet bowl cleaning system 100 to use the proper cleaning tools to perform spot cleaning. Figure 7 shows a pipeline to train a perception model for stain detection, for example, from synthetic images. In step 702, the perception pipeline builder module 426 analyses an image to detect presence of stains in a toilet bowl. In step 704, the toilet bow'l image is segmented into its respective parts, as trained to be recognised by the perception pipeline builder module 426. In step 706, the stains are fitted into bounding boxes. In step 708 the bounding boxes are mapped to the segmented toilet bowl image from step 704, where majority vote is then used to decide which toilet bowl part the stain lies. The perception pipeline builder module 426 thus provides an algorithm to differentiate between parts of a toilet bowl and detect the toilet bowl part on which a stain is located.

[0069]

[0070]

[0049] Figure 8 shows high-fidelity synthetic images 802 generated from a robotics developer simulation platform and reference application used to train the algorithm. In one implementation (refer Figure 11), the simulation platform is hosted in a separate terminal 1110 in communication 1104 with the robot software platform 400, although the simulation platform may also be hosted in the robot software platform 400. The simulation platform is advantageous for its capability to generate varied data sets (various lightings 806. floor textures (such as different materials 808 and colours 810), stain colors and size); perform automatic generated labelling; and facilitating faster training-to-deployment lead time. Using such a platform or from publicly available database images, the algorithm can be trained with images of toilet bowls with stains and peripheral objects labelled. The training images 804 may be pictures or synthetic; and may be taken with any one or more of the following conditions: different lightings, floor texture, stain colour and size.

[0071]

[0050] By feeding the algorithm with sufficient corpus images, Al models for the toilet bowl cleaning system 100 can be trained to do the following (a) stain detections; (b) toilet seat cover status detection; (c) identify and locate faeces / urinal stains; (d) spot cleaning (detect remaining stains and clean again); (e) scene understanding (precise alignment with respect to a toilet bowl); (f) anomaly detections (e.g. tissue paper on toilet bowl, choked toilet bowl, flooded toilet bowl). The trained Al models are stored and retrievable from the Al library module 420 (see Figure 4) and can be distributed to load into computing platform 116 of toilet bowl cleaning system 100.

[0072]

[0051] The navigation stack module 428 provides the base software for localisation, mapping and navigation functionality; when deployed to the toilet bowl cleaning system 100, allows the base 102 to autonomously navigate between toilets to reach an assigned toilet, navigate within the toilet and manoeuvre in and out of a toilet cubicle. The navigation is facilitated by analysing the received images from the camera assembly 108 for presence of path markers for use in guiding the base 102 to the toilet bowl. The navigation stack module 428 also causes the base 102 to return to a docking station when a cleaning task is completed. The simulation module 430 of the mobile robot base module 106 allows testing and validation of new navigation capabilities, before deployment into the base 102.

[0073]

[0052] Figure 9 shows several internal components of the base 102 of the toilet bowl cleaning system 100. A router 902 allows for wireless communication of the computing platform 116 with the remotely located robot software platform 400, such as to receive instructions from the deployment platform module 402. An arm controller box 906 is used to control the movement of the robotic arm 104. The arm controller box 906 includes a compliant controller to adjust contact force the robotic arm 104 uses with a cleaning head attachment 110. The compliant control enables the robot arm 104 to perform tasks involving contact-based cleaning. The compliant controller automatically adjusts the contact force when the cleaning head attachment 110 touches a surface. This automatic adjustment compensates for potential positioning errors, ensuring the cleaning head attachment 110 establishes firm contact with the surface being cleaned for cleaning effectiveness.

[0074]

[0075]

[0053] A navigation module 920 provides navigation control to the base 102 as described in Figure 4 and is thus not elaborated. The navigation module 920 works in tandem with LIDAR sensors 916 which are used to prevent the base 102 from colliding with obstacles. DC power converters 908 provide DC power supply for the robot arm 104, Al computers, networking devices (such as the router 902), accessories (water tanks, nozzles, etc) and motor drive 910. The motor drive 910 drives a wheel assembly which allows the base 102 to move.

[0076]

[0054] Figure 10 is a perspective view of a tool rack 1002, located at the base 102, which provides storage for the cleaning head attachments 110. Brush heads of the cleaning head attachments 110 are omitted for the sake of simplicity and for a clearer view of the storage area.

[0077]

[0055] In one implementation, the tool rack 1002 is disposed at a waste water tank 1006 into which the cleaning head attachments 110 are introduced after a cleaning task is completed. The tool rack 1002 has one or more disinfecting nozzles 1004 with an end disposed proximate to the waste water tank 1006 and another end coupled to draw from a detergent tank. The one or more disinfecting nozzles 1004 wash the cleaning head attachments 110 by spraying detergent to remove debris and disinfect the cleaning head attachments 110.

[0078]

[0056] The detergent tank from which the one or more disinfecting nozzles 1004 draws is located at the base 102 and may also be a source of detergent for the cleaning nozzle 244 shown in Figure 2. In such an implementation, one end of the cleaning nozzle 244 is disposed proximate to where the cleaning head attachment 110 couples and the other end coupled to draw from the detergent tank when performing a cleaning task.

[0079]

[0057] From the above discussion for Figures 4, 5 and 8, the toilet bowl cleaning system 100 can perform the following tasks in any order:

[0080]

[0058] I. Brush a toilet bowl part based on predefined motion planning, or motion constructed in realtime from pre-defined motion

[0059] 2. Flush the toilet by pressing the toilet flush button

[0060] 3. Lift I close the toilet seat cover

[0061] 4. Spray clean water into the toilet bowl

[0062] 5. Spray detergent into the toilet bowl

[0063] 6. Spray detergent into the waste water tank

[0064] A possible cleaning sequence which the toilet bowl cleaning system 100 is as follows, with the the sequence being reconfigurable by the cleaning task management and execution module 502 (see Figure 5):

[0081] a. When assigned to a cleaning task, the base 102 will move automatically into a cleaning position in a toilet cubicle

[0082]

[0083] b. Perception Al will be used to check the status of the toilet bowl. If the toilet bowl is cleanable, it will start the cleaning process. Otherwise, the toilet bowl cleaning system 100 will send analert to ask for human intervention. This could occur, for example, when there are anomaly situations, such as presence of choke, breakage or toilet bowl flooding.

[0084] c. The robotic arm 104 will flush the toilet by using its retractable prod (an extendable rod mounted on the End of Arm Tooling (EOAT))

[0085] d. The robotic arm 104 will active the nozzle on the EOAT to spray detergent into the toilet bowl c. The robotic arm 104 will couple to a first cleaning head attachment suitable for cleaning the inside of the toilet bowl

[0086] f. The robot arm 104 will move to a position to use perception to detect and perform targeted cleaning if any

[0087] g. The robot arm 104 will return the first cleaning head attachment to tool rack and couple to a second cleaning head attachment suitable for cleaning a toilet cover

[0088] h. The robotic arm 104 will use the second cleaning head attachment to close the toilet cover to clean its top surface, then opens the toilet cover to clean its back surface. Subsequently, the robotic arm 104 cleans the top surface of the toilet seat cover, then opens the toilet seat cover to clean its back surface. Next, the robotic arm 104 cleans the top surface of the toilet bowl rim. Finally clean the external surface of the toilet bowl

[0089] i. The robotic arm 104 will move to a position to use perception to detect and perform targeted cleaning if any on the back surface of the seat cover and top surface of the toilet bowl rim j. The robotic arm 104 will return the second cleaning head attachment and couple to a third cleaning head attachment suitable for a drying operation

[0090] k. The robot arm 104 will use the third cleaning head attachment to wipe dry the back surface of the seat cover, the top surface of the toilet bowl rim, the external surface of toilet bowl. Subsequently, the robotic arm 104 uses the third cleaning head attachment to close the toilet cover and wipe dry the top surface of the toilet cover. Then open toilet cover, use the third cleaning head attachment to wipe dry the back surface of the toilet cover and the top surface of the toilet seat cover.

[0091] l. The robotic arm 104 finishes cleaning its assigned toilet bowl and the base 102 will navigate to the next cubicle.

[0092] m. The toilet bowl cleaning system 100 will clean the toilet bowls in each of the rest of the cubicles and return to the docking station to wait for the next cleaning task.

[0093]

[0065] In the application, unless specified otherwise, the terms "comprising", "comprise", and grammatical variants thereof, intended to represent "open" or "inclusive" language such that they include recited elements but also permit inclusion of additional, non-explicitly recited elements.

[0094]

[0066] While this invention has been described with reference to exemplary embodiments, it will be understood by those skilled in the art that various changes can be made and equivalents may be substituted for elements thereof, without departing from the scope of the invention. In addition, modification may be made to adapt the teachings of the invention to situations and materials, without

[0095]

[0096] departing from the essential scope of the invention. Thus, the invention is not limited to the examples that are disclosed in this specification but encompasses all embodiments falling within the scope of the appended claims.

[0097]

Claims

CLAIMS1. A toilet bowl cleaning system comprising:a robotic arm;a base to which the robotic arm is coupled, the base comprising a plurality of cleaning head attachments for the robotic arm;a camera assembly; anda computing platform configured toanalyse received images from the camera assembly for presence of a toilet bowl with an algorithm trained to differentiate between parts of a toilet bowl and detect the toilet bowl part on which a stain is located;select, based on the stain location, a cleaning head attachment for use to spot clean the stain from the plurality of cleaning head attachments; andtransmit a command to the robotic arm to couple with the selected cleaning head attachment for spot cleaning the stain.

2. The toilet bowl cleaning system of claim 1, wherein the computing platform is further configured to:verify for stain removal from images received from the camera assembly after the robotic arm has cleaned the stain; andtransmit a command to re-clean the stain or for human intervention, in response to detecting that the stain remains.

3. The toilet bowl cleaning system of any one of the preceding claims, wherein the computing platform is further configured to:analyse the received images for presence of a marker arrangement; andquery a database with data extracted from the marker arrangement to retrieve information about the toilet bowl configuration.

4. The toilet bowl cleaning system of claim 3, wherein the toilet bowl configuration comprises any one or more of: number of toilet bowl parts; measurements for each toilet bowl part; and assembly arrangement of the toilet bowl parts.

5. The toilet bowl cleaning system of claim 3 or 4, wherein the marker arrangement comprises two parts during calibration, a permanent part located on a fixture in a vicinity of the toilet bowl; and a temporary part on the toilet bowl to locate the toilet bowl relative to the permanent part, the temporary part being removed after calibration.

6. The toilet bowl cleaning system of claim 5, wherein the computing platform is further configured to, in use, factor the calibration from the marker arrangement when moving the robotic arm to clean one or more of: the stain or any other toilet bowl part.

7. The toilet bowl cleaning system of any one of the preceding claims, wherein the computing platform is further configured to:reference one or more libraries for selection of a cleaning motion for the robotic arm based on any one or more of: the stain location, the toilet bowl configuration and surroundings of the toilet bowl; andtransmit a command for the robotic arm to execute the selected cleaning motion.

8. The toilet bowl cleaning system of any one of the preceding claims, further comprising a compliant controller to adjust contact force the robotic arm uses with the cleaning head attachment.

9. The toilet bowl cleaning system of any one of the preceding claims, wherein the computing platform is further configured to: analyse the received images for presence of path markers for use in guiding the toilet bowl cleaning system to the toilet bowl.

10. The toilet bowl cleaning system of any one of the preceding claims, wherein the computing platform is further configured to:analyse the received images for presence of anomaly situations; andtransmit a signal for human intervention in response to detection of one or more anomaly situations.

11. The toilet bowl cleaning system of claim 10, wherein the anomaly situations comprise any one or more of: presence of a choke, breakage.

12. The toilet bowl cleaning system of any one of the preceding claims, wherein the computing platform is further configured to: return the toilet bowl cleaning system to a docking station when cleaning task is completed.

13. The toilet bowl cleaning system of any one of the preceding claims, wherein the algorithm is trained with images of toilet bowls with stains and peripheral objects labelled.

14. The toilet bowl cleaning system of claim 13, wherein the training images comprise images having any one or more of the following conditions: different lightings, floor texture, stain colour and size.

15. The toilet bowl cleaning system of claim 14, wherein the training images are pictures or synthetic.

16. The toilet bowl cleaning system of any one of the preceding claims, wherein the camera assembly is located on the robotic arm, proximate to where the selected cleaning head attachment couples.

17. The toilet bowl cleaning system of any one of the preceding claims, further comprising: a detergent tank; anda cleaning nozzle with one end disposed proximate to where the selected cleaning head attachment couples and another end coupled to draw from the detergent tank when performing a cleaning task.

18. The toilet bowl cleaning system of claim 17, further comprisinga waste water tank into which the selected cleaning head attachment is introduced when a cleaning task is completed; andone or more disinfecting nozzles with each end disposed proximate to the waste water tank and another end coupled to draw from the detergent tank to wash the selected cleaning head attachment.

19. The toilet bowl cleaning system of any one of the preceding claims, wherein one or more of cleaning head attachments is motorised; configured to draw power when coupled with the robotic arm or when stored.

20. The toilet bowl cleaning system of any one of the preceding claims, wherein the base comprises storage for each of the cleaning head attachments.

21. The toilet bowl cleaning system of any one of the preceding claims, wherein the robotic arm further comprises a retractable prod to actuate the toilet bowl flush.

Images