Closestool cleaning robot control method and system

Abstract

The invention relates to the technical field of manipulators, and discloses a closestool cleaning robot control method and system.The closestool cleaning robot control method comprises the steps that visual information is synchronously collected through double cameras, and the tail end posture and joint state data of a mechanical arm are obtained; calling a three-layer cleaning decision model based on visual input, and dynamically loading an action generation model corresponding to a cleaning level; on the basis of the image and the tail end posture data, a tail end movement track and an electromagnet control instruction are generated in real time through a corresponding track generation model, and a mechanical arm is driven to execute a wiping action after continuous multi-frame verification; and judging the cleaning completion degree according to the image difference and the brightness threshold, maintaining the cleaning cycle of the current layer if the cleaning completion degree does not reach the standard, and switching to the next cleaning layer after the cleaning completion degree reaches the standard. The problems of high noise, frequent maintenance, large size and high cost in the prior art are solved, and the purposes of simplifying maintenance, reducing noise, reducing size and improving cleaning efficiency are achieved.

Description

Technical Field

[0001] This invention relates to the field of work transportation technology, and in particular to robotic arms. Background Technology

[0002] Toilet cleaning is an unavoidable daily task. People of all ages inevitably accumulate stains on its surface during use. When cleaning becomes a necessity, is there a tool that, in today's rapidly developing technological world, can replace manual labor, freeing up manpower, time, and physical discomfort to clean the toilet? Toilet cleaning tools, or rather solutions, aim to answer this question. However, most toilet cleaning tools on the market are ineffective, resulting in low adoption rates. Automatic toilets are not only expensive but can only clean the inner surface, not the seat or lid. Advanced cleaning robots exist abroad, but they are expensive, complex to use, and bulky, making them suitable only for commercial bathrooms, not the confined spaces of home bathrooms. Furthermore, their control processes require manual intervention, lacking intelligent cleaning capabilities and the ability to determine whether cleaning is complete; they simply follow a procedure, leading to questionable cleaning effectiveness. Built-in toilet cleaning tools have limited cleaning coverage, incur installation costs, and their integrated design raises questions for users about "how to self-clean" or "who will clean the cleaning tool." Overall, there is currently no truly satisfactory toilet cleaning robot on the market. In light of the above, several drawbacks can be summarized: 1. Size and Weight: Existing cleaning robots are large in size and heavy, with large 7-joint robotic arms. This size limits their use to public or commercial restrooms. 2. High Cost: Current cleaning robots are only available through rental, at prices far beyond the reach of most families. 3. Complex Positioning / Sensing Systems: Positioning solutions relying on laser SLAM, radar, or 3D scanners are costly in hardware, complex to debug and maintain, and susceptible to interference from mirrors and water mist. 4. Noise and Cleaning Efficiency: Cleaning robots often use vacuum cleaners, resulting in low cleaning efficiency and noise, which can be inconvenient in a home environment. Their cleaning effectiveness is limited on damp or uneven surfaces. 5. Installation Costs: Unlike robotic arm solutions, toilet cleaning tools are embedded in the toilet bowl. However, this installation method is not suitable for every toilet and incurs installation costs. 6. Cleaning coverage: Other automatic cleaning solutions (including automatic flushing and toilet cleaning) can only cover the cleaning of the inner wall of the toilet, but cannot cover the outer surface of the toilet, the inner and outer surfaces of the seat, and the inner and outer surfaces of the toilet lid. When men use the toilet, the outer surface of the toilet base is often the easiest to get dirty, and the current technology cannot clean it.

[0003] For example, Chinese patent CN114129084A discloses a dual-robotic arm mobile robot working system and collaborative working method for toilet cleaning, providing the following technical solution: This invention provides a dual-robotic arm mobile robot working system and collaborative working method for toilet cleaning, including host computer software, a power module, a dual-robotic arm device, a main processor module, a navigation system module, and a vision system module; the main processor module is connected to the host computer software, the power module, the dual-robotic arm device, the navigation system module, and the vision system module respectively; the vision system module is connected to a vision camera. This invention proposes a toilet cleaning robot solution; minimizing labor costs and replacing manual cleaning work. However, the aforementioned dual-robotic arm mobile robot working system and collaborative working method for toilet cleaning, using dual robotic arms and a complex navigation system, is bulky, expensive, easily affected by environmental interference, and relies on reusable tools, leading to the risk of secondary pollution. It cannot adapt to the small spaces of a home, has a low level of intelligence, and lacks AI-driven adaptive cleaning. Summary of the Invention

[0004] This invention solves the problems of high noise, frequent maintenance, large size and high cost in the prior art, and proposes a control method and system for a toilet cleaning robot, which achieves the purpose of simplifying maintenance, reducing noise, reducing size and improving cleaning efficiency.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: A method for controlling a toilet cleaning robot includes: Visual information is collected synchronously by dual cameras, and data on the posture and joint status of the robotic arm's end effector are obtained. Based on visual input, a three-layer cleaning decision model is invoked, and the corresponding action generation model for the cleaning level is dynamically loaded. Based on image and end-effector posture data, the motion trajectory of the end-effector and electromagnet control commands are generated in real time through the corresponding trajectory generation model. After verification for several consecutive frames, the robotic arm is driven to perform the wiping action. The cleaning completion level is determined based on image difference and brightness threshold. If the standard is not met, the current layer cleaning cycle is maintained. Once the standard is met, the process switches to the next cleaning layer.

[0006] The simplified positioning and sensing system avoids reliance on expensive laser SLAM or depth radar, maintaining stability and reliability in humid, mirror-reflective bathroom environments. At the same time, the AI-controlled Diffusion model generates action sequences in real time, ensuring the intelligence and efficiency of the cleaning process, reducing noise and maintenance costs, and eliminating secondary pollution.

[0007] As a preferred approach, the three-layer cleaning decision model divides the action of cleaning the toilet into several cleaning action sequences. Each cleaning action sequence is trained using collected training data, allowing the model to determine whether to repeat the cleaning, perform the previous action, or proceed to the next action label under the current visual input. The training data is then labeled with numbers corresponding to the cleaning action sequences, and several data augmentation methods are randomly applied to the training data.

[0008] Data augmentation enhances the model's generalization ability, adapting to deviations in different household toilet environments. This ensures the robustness of cleaning decisions, avoids cleaning failures due to environmental changes, and optimizes cleaning efficiency and coverage through intelligent switching of action sequences.

[0009] Preferably, the input data for the trajectory generation model includes: a four-channel tensor stitched together from the synchronous images of the dual cameras at the current and previous moments, and a twenty-dimensional vector formed by merging the six-dimensional representation of the robot arm end pose and rotation at the current and previous moments with the gripper state; the four-channel tensor and the twenty-dimensional vector are then input after being fused by an observation encoder.

[0010] This input design makes trajectory generation more accurate and continuous, captures complete contextual information, and reduces motion errors; by observing the encoder to fuse data, the inference effect of the Diffusion model is optimized, ensuring smooth and efficient wiping actions, and improving cleaning quality and efficiency.

[0011] As a preferred option, the trajectory generation model generates the motion trajectory of the end effector and the electromagnet control commands in real time, specifically including: the motion prediction model based on the Transformer architecture outputs the current state noise, generates 16 future actions by denoising through 10 step iterations, executes the first 8 actions, and uses the last 8 actions as a timing buffer.

[0012] It improves real-time response by reducing computational latency by executing only the first 8 actions; the timing buffer mechanism enhances adaptability in dynamic environments, ensuring stable and reliable cleaning actions and avoiding cleaning interruptions caused by sudden changes.

[0013] Preferably, the system uses dual cameras to simultaneously acquire visual information, identifies the toilet mask area in the camera image using a visual model, calculates the average depth of the mask area using depth camera data, and prompts the user to adjust the robot's position if the average depth is greater than the working radius of the robotic arm.

[0014] This avoids the problem of the robotic arm being unable to reach the target location. Through depth calculation and mask recognition, it improves the reliability and user-friendliness of deployment. It also prompts users to adjust the position in a timely manner, reducing the risk of cleaning failure and ensuring the applicability of the system in small home environments.

[0015] Preferably, the trajectory generation model includes several independent sub-models, each corresponding to a cleaning action sequence, with a consistent architecture. The training data is allocated according to the task complexity, specifically including more than 100 teleoperation teaching data for low-complexity tasks with a training period of less than 24 hours, and more than 1000 teleoperation teaching data for high-complexity tasks with a training period of 3-7 days. The electromagnet control commands include execution commands with consistent predicted states for 3 consecutive frames, and determination of whether the duration of a single power-on exceeds 30 seconds, with automatic power-off alarm in case of timeout.

[0016] The sub-models allocate data according to task complexity, optimizing training efficiency and performance and ensuring accuracy for high-difficulty cleaning tasks; the electromagnet control prevents misoperation and equipment overheating through continuous frame verification and timeout power-off, improving safety and reliability and reducing maintenance costs.

[0017] Preferably, determining the cleaning completion level based on image difference and brightness threshold includes: calculating the difference matrix using partitioned block sampling, extracting suspected dirty areas, statistically analyzing the brightness histogram of suspected dirty areas, and determining that the cleaning is not up to standard if the standard deviation or mean does not meet the requirements; the calculation of the difference matrix using partitioned block sampling specifically includes: dividing the cleaning area into a grid, starting the difference calculation for the grid blocks covered by the robotic arm end-effector trajectory, and dynamically ignoring reflective areas.

[0018] The zoned grid and dynamic reflection-ignoring features provide more accurate dirt detection and reduce false positives; brightness histogram analysis ensures the objectivity of cleaning quality assessment, avoids the need for manual review, and improves cleaning thoroughness and efficiency.

[0019] A toilet cleaning robot control system includes: a robotic arm and a dual-camera vision acquisition module. The robotic arm includes several rotary joints. The dual-camera vision acquisition module is respectively deployed on the base and end of the robotic arm. The end of the robotic arm is provided with a replaceable actuator module. The base is connected to a mobile chassis module.

[0020] This lightweight design reduces size and weight, making it suitable for the small spaces of family bathrooms; dual cameras deployed at the base and end ensure all-around visual coverage, and replaceable actuator modules increase flexibility and reduce deployment and maintenance costs.

[0021] Preferably, the actuator module includes a gripper and an electromagnet module, wherein the electromagnet module has an electromagnet control unit and an electromagnet support, the electromagnet support is connected to the robotic arm and the electromagnet control unit, and the electromagnet module adsorbs disposable magnetic cleaning sheets.

[0022] The combination of an electromagnet and disposable cleaning pads eliminates secondary pollution and simplifies maintenance; the support and connection design ensures easy and reliable installation, avoids the risk of scratching the toilet surface with traditional grippers, and reduces user costs through low-cost consumables.

[0023] Preferably, the rotary joint includes: a first joint located on the base, controlling the main arm to rotate left and right in the horizontal direction; a second joint connecting the base and the main arm, controlling the main arm to bend in the vertical direction; a third joint connecting the main arm and the secondary arm, controlling the secondary arm to bend in the vertical direction; a fourth joint and a fifth joint connecting the functional block, wherein the fourth joint rotates in any direction and the fifth joint controls the bending in the vertical direction; and a sixth joint connecting the functional block and the end cap, controlling the end cap to rotate in any direction.

[0024] This joint configuration provides ample freedom of movement, optimizing the range of cleaning motions; it ensures that the robotic arm covers all areas of the toilet, improving cleaning precision and efficiency, while the speed-limiting design guarantees smooth and safe movement.

[0025] Compared with the prior art, the beneficial effects of the present invention are as follows.

[0026] 1. This invention utilizes a dual-camera vision solution, significantly reducing hardware costs and avoiding interference from humid mirror environments, ensuring a stable and reliable cleaning process. Simultaneously, electromagnet control commands undergo continuous multi-frame verification and a timeout power-off mechanism, reducing the risk of misoperation and preventing overheating damage. This design simplifies the positioning and sensing system, significantly reduces the overall size, and allows the robot to be easily deployed in small home bathrooms, reducing noise and maintenance frequency at the source and improving user comfort.

[0027] 2. This invention is based on a three-layer cleaning decision model and a trajectory generation sub-model. The system dynamically loads action sequences and adapts to different toilet environment deviations through data augmentation technology, improving generalization ability. Input is fused from dual-camera images and robotic arm posture data, which are optimized by an observation encoder to generate a smooth trajectory. Frame-by-frame actions and a buffering mechanism enhance real-time responsiveness. This not only ensures precise and efficient cleaning actions but also reduces the need for manual intervention through intelligent dirt detection and brightness histogram analysis, achieving full-coverage cleaning and significantly improving overall efficiency and quality.

[0028] 3. The lightweight robotic arm of this invention, combined with replaceable actuator modules, simplifies installation and maintenance processes, completely avoiding the risk of secondary contamination and surface scratches. The joint configuration provides ample freedom of movement, covering all areas of the toilet, while the speed-limiting design ensures smooth and safe movement. Dual cameras provide all-around visual coverage, combined with a mobile chassis, enhancing flexibility. Users only need low-cost consumables to complete cleaning. The overall design fits the needs of the home environment, optimizing ease of use and economy. Attached Figure Description

[0029] Figure 1 This is an overall flowchart of a toilet cleaning robot control method and system according to the present invention.

[0030] Figure 2 This is a schematic diagram of the robotic arm structure of a toilet cleaning robot control method and system according to the present invention.

[0031] Illustration: 1. First joint; 2. Second joint; 3. Third joint; 4. Fourth joint; 5. Fifth joint; 6. Sixth joint; 7. Electrical panel; 8. Main arm; 9. Secondary arm; 10. Functional block. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be further described in detail below with reference to the accompanying drawings. The proportions of the components are not drawn to scale, and the proportions and dimensions shown in the drawings should not limit the essential technical solutions of the invention. These embodiments do not exhaustively describe all details, nor do they limit the invention to merely the specific embodiments described.

[0033] See Figure 1-2 As shown, a method for controlling a toilet cleaning robot includes: Visual information is collected synchronously by dual cameras, and data on the posture and joint status of the robotic arm's end effector are obtained. Based on visual input, a three-layer cleaning decision model is invoked, and the corresponding action generation model for the cleaning level is dynamically loaded. Based on image and end-effector posture data, the motion trajectory of the end-effector and electromagnet control commands are generated in real time through the corresponding trajectory generation model. After verification for several consecutive frames, the robotic arm is driven to perform the wiping action. The cleaning completion level is determined based on image difference and brightness threshold. If the standard is not met, the current layer cleaning cycle is maintained. Once the standard is met, the process switches to the next cleaning layer.

[0034] A toilet cleaning robot control system includes: a robotic arm and a dual-camera vision acquisition module. The robotic arm includes several rotary joints. The dual-camera vision acquisition module is respectively deployed on the base and end of the robotic arm. The end of the robotic arm is provided with a replaceable actuator module. The base is connected to a mobile chassis module.

[0035] This invention achieves low-cost, disposable cleaning, automatic dirt disposal, and user-customizable cleaning methods for toilets by using a "lightweight single-arm + low-requirement mobile chassis + binocular vision + disposable cleaning consumables solution + replaceable end-point design + artificial intelligence full-process control". It can better solve the problem of toilet cleaning and overcome the shortcomings of the prior art.

[0036] like Figure 1 In one embodiment shown, Figure 1 This is an overall flowchart of a toilet cleaning robot control method and system according to the present invention. The toilet cleaning robot control method first acquires visual information synchronously through dual cameras and obtains the posture and joint state data of the robotic arm end effector; then, based on the visual input, it calls a three-layer cleaning decision model. This model divides the cleaning action into several cleaning action sequences, and trains each sequence using data-augmented training data, enabling the model to determine whether to repeat cleaning, execute the previous action, or proceed to the next action.

[0037] Then, a trajectory generation model is input based on image and end-effector posture data. This model first receives a four-channel tensor stitched from synchronized images from dual cameras and a twenty-dimensional vector merging the robotic arm's state. After being fused by an observation encoder, the two are input into a motion prediction model based on a Transformer architecture. Then, 16 future actions are generated through 10-step iterative denoising. The first 8 actions are executed and the last 8 actions are used as a timing buffer. At the same time, electromagnet control commands are generated. The electromagnet control commands are only executed if the predicted state is consistent for 3 consecutive frames. If a single power-on exceeds 30 seconds, an automatic power-off alarm is triggered.

[0038] Meanwhile, during the dual-camera acquisition process, a visual model identifies the toilet mask area, and the average depth value is calculated by combining it with depth data. If the depth exceeds the working radius of the robotic arm, the user is prompted to adjust their position. The trajectory generation model contains multiple independent sub-models, each corresponding to a cleaning action sequence and sharing a consistent architecture. Training data is allocated according to task complexity: low-complexity tasks use over 100 teleoperation teaching data points with a training period of less than 24 hours, while high-complexity tasks use over 1000 data points and are trained for 3-7 days.

[0039] Finally, the cleaning completion is determined based on image difference and brightness threshold: First, the cleaning area is divided into a grid, and difference calculation is started only for the blocks covered by the trajectory of the robotic arm end to extract suspected dirty areas, dynamically ignoring reflective areas; then, the brightness histogram of suspected dirty areas is calculated. If the standard deviation or mean does not meet the requirements, it is determined that the standard has not been met, and the cleaning cycle of the current layer is maintained until the standard is met before switching to the next cleaning level.

[0040] In one embodiment, the specific process steps are as follows: S1. User Startup The initial step of the process is initiated by the user to activate the device.

[0041] S2. Device Startup / Sensor Activation The robot system is powered on, and sensors such as cameras and robotic arms are initialized.

[0042] S3. Self-test The system performs a hardware self-test; if it fails, it returns to the S1 restart process.

[0043] S4.1 Environmental Assessment Detecting the toilet's position and the robotic arm's working range: Enter S5 via →; Failed → Redirect to S4.2.

[0044] S4.2 voice prompts the user to adjust the machine position. After issuing a voice prompt to guide the user to reposition the robot, return to S4.1 for another assessment.

[0045] S5. Perform cleaning The three-layer cleaning process has officially begun: First layer of cleaning (toilet seat): S6.1 Clean the outer surface of the toilet seat; S6.2 External Surface Stains Judgment → If not up to standard (N), clean again; if up to standard (P), continue. S6.3 Open the toilet seat; S6.4 Clean the inside of the toilet seat; S6.5 Inner wall stain assessment → If not up to standard (N), clean again; if up to standard (P), continue.

[0046] Second layer of cleaning (seat ring): S7.1 Clean the outer surface of the seat ring; S7.2 External Surface Stains Judgment → If not up to standard (N), clean again; if up to standard (P), continue. S7.3 Open the toilet seat; S7.4 Clean the inner wall of the seat ring; S7.5 Inner wall stain assessment → If not up to standard (N), clean again; if up to standard (P), continue.

[0047] Third layer of cleaning (base): S8.1 Clean the surface of the toilet seat; S8.2 Surface stain assessment → If not up to standard (N), clean again; if up to standard (P), continue. S8.3 Close the seat ring; S8.4 Close the toilet seat.

[0048] First, the system needs three main AI models. Model 1 is used to determine whether the current position is feasible, or whether the relative position of the robot's robotic arm and the toilet is appropriate (if it is too far away, it cannot reach the toilet). Model 2 is used to determine the current execution state (depending on the degree of dirt, whether to continue this round of wiping or to the next wiping action). Model 3 is the most crucial part, which is the model that directly drives the robotic arm to complete the wiping work, and the output is an action sequence.

[0049] Model 1: Mask R-CNN (ResNet-50-FPN) was used here. It is a common vision model. It was trained by fine-tuning on the COCO dataset and 500 toilet sample images collected by ourselves. The model can accurately identify the toilet in the test set (accuracy of over 99%) and output the pixel mask of the toilet in the output image.

[0050] import torch,torchvision import Image from PIL from torchvision.transforms import functional as F #1) Load the model model=torchvision.models.detection.maskrcnn_resnet50_fpn(weights="DEFAULT")model.eval() #2) Read the graph and convert it to a tensor img=Image.open("bathroom.jpg").convert("RGB") tensor = F.to_tensor(img) #3) Reasoning with torch.no_grad(): out = model([tensor])[0] # Batch size only = 1 #4) Filter out "toilet" (category_id=70) keep=[i for i,c in enumerate(out["labels"])if c.item()==70and out["scores"][i]>0.5]toilet_masks=out["masks"][keep]#(N,1,H,W) probability mask.

[0051] The robot uses the RealSense camera's library function (pyrealsense2) to retrieve depth information from the image and calculate the average depth of probability, which represents the distance between the target toilet and the robot. When the distance d exceeds the working radius of the robotic arm (626.75mm), the user is prompted to "move the robot closer to the toilet".

[0052] Model 2: Model 2 is a classification model fine-tuned based on ViT-Small. It requires 11 classification classes and was trained using nearly 5000 images (approximately 400-500 images per state). The model determines, under the current visual input, whether to repeat the cleaning process, perform the previous action, or proceed to the next action label. Based on the action of cleaning the toilet, it is segmented into 11 states as follows: 0-> Clean the outer surface of the toilet seat 1-> Open the toilet lid 2-> Clean the inside of the toilet seat 3-> Clean the outer surface of the seat ring 4-> Open the toilet seat 5-> Clean the inner surface of the seat ring 6-> Clean the toilet base 7-> Close the seat ring 8-> Close the toilet lid 9->Flush 10->Return to initial state The collected training data consists of each photo labeled with a number from 0 to 10, representing the action the robotic arm should take when it sees these images. Considering that the user's home toilet or environment may differ from the laboratory environment, the training data is randomly augmented before training.

[0053] The methods for data augmentation are as follows: 1. Random scaling + cropping 2. Horizontal flip 3. Slight rotation / perspective 4. Color jitter 5. Choose between Gaussian blur or sharpen. 6. Simulate actual occlusion 7. Random noise.

[0054] The code implementation is as follows: from torchvision.transforms import v2 as T train_tf = T.Compose([ T.RandomResizedCrop(224,scale=(0.8,1.0),ratio=(0.95,1.05),antialias=True), T.RandomHorizontalFlip(p=0.5), T.RandomAffine(degrees=7,translate=(0.02,0.02),shear=5,p=0.3), T.ColorJitter(0.3,0.3,0.2,0.05,p=0.8), T.RandomApply([T.GaussianBlur(3,(0.1,1.5))],p=0.2), T.RandomAdjustSharpness(1.5,p=0.2), T.RandomErasing(scale=(0.02,0.15),ratio=(0.3,3.3),p=0.25), T.ToTensor(), T.Normalize(mean=[0.5]*3,std=[0.5]*3), ]).

[0055] Model 3 is the core of this invention. It is responsible for inferring the future trajectory (a sequence of 8 actions) from the current and historical visual input and the robotic arm posture input. This trajectory will be regarded as a specific cleaning sequence.

[0056] The model's input includes an image encoder (two cameras, so four 2x2 images, which will be resized to 480*640*3) with two time steps (if the master controller infers at 10Hz, this means the state at two time points separated by 0.1s) and a low-DMI (x, y, z, euler, gripper, where euler is converted to rotation6d, so the total dimension is (3+6+1)*2=20). The concatenated data is passed to an observation encoder, and then to an action prediction model designed by a transformer for trajectory generation. The principle of trajectory generation is that the denoise module denoises the noise in the current time state predicted by the model, and the trajectory is generated by 10 denoise steps (100 during training and 10 during inference, because in practice, 10 steps can achieve more than 90% optimal performance). The Denoise action has a length of 16, but only the first 8 are executed. This is because the principle of model generation can be understood as the model reasoning about the information of the current and past frames to infer the trajectory and actions that the model believes will occur in the next 16 time periods. However, since the reasoning is a time-series signal, the closer the action is to the current time, the more timely it is theoretically. Therefore, only the first eight actions are taken as the actual actions sent to the robot for execution.

[0057] The data used in Model 3 is obtained through teleoperation: 1. Camera video data; 2. End-effector posture of the robotic arm; 3. Target posture sent by teleoperation. For different scenarios (referring to each action point of toilet cleaning), the data required for model training and the training time are also different. For example, the task of flipping the toilet seat is relatively simple, using only 100 demonstrations (referring to data collection by controlling the robotic arm with teleoperation), and training can be completed in one night (24 hours). However, the task of wiping the toilet seat is more difficult, using more than 1,000 demonstration data. Similarly, due to the larger spatial distribution of the data and the increase in the number of data, the training time will also be longer, usually requiring 3 days to a week of training (single 3090 graphics card).

[0058] Model 3 is simply a general term for the types of models used. In this project, each toilet cleaning action sequence requires a specific Model 3 to complete. That is to say, in the 11 action sequences from 0 to 10, 11 Model 3s with the same architecture but different training data and different uses were trained. Finally, in the main control unit, a certain Model 3 is selected and loaded to complete the cleaning task based on the prediction of Model 2.

[0059] like Figure 2 In one embodiment shown, Figure 2 This is a schematic diagram of the robotic arm structure of a toilet cleaning robot control method and system according to the present invention. A toilet cleaning robot control system includes: a robotic arm and a dual-camera vision acquisition module. The robotic arm includes several rotary joints. The dual-camera vision acquisition module is respectively deployed on the base and end effector of the robotic arm. A replaceable actuator module is provided at the end effector of the robotic arm. The base is connected to a mobile chassis module. The actuator module includes a gripper and an electromagnet module. The electromagnet module has an electromagnet control unit and an electromagnet support. The electromagnet support connects the robotic arm and the electromagnet control unit. The electromagnet module adsorbs disposable magnetic cleaning sheets. The rotary joints include: The first joint, located on the base, controls the left and right rotation of the main arm in the horizontal direction; The second joint connecting the base and the main arm controls the vertical bending of the main arm; The third joint connecting the main arm and the secondary arm controls the vertical bending of the secondary arm; The fourth and fifth joints of the connecting functional block can rotate in any direction, and the fifth joint controls bending in the vertical direction; The sixth joint connects the functional block to the end effector, controlling the end effector to rotate in any direction.

[0060] In one embodiment, this invention employs a solution where cleaning control is largely entrusted to AI control. Simply put, the cleaning actions performed by the robotic arm are generated by artificial intelligence, a technology based on Diffusion trajectory generation. Previously, this technology was only used in laboratory environments for gripping, moving, and classifying objects; no one had attempted its application in specific scenarios like cleaning toilets. The principle of the Diffusion model aligns with the data-driven AI paradigm, that is, by fitting probabilistic models for various situations using multi-sample data, the final model can infer the future spatial position of the robotic arm from inputs (camera footage, robotic arm spatial position). The algorithm of this invention is a derivative of the generative model framework DDIM (Diffusion is also based on DDIM). Data pairs consist of (model-generated actions, actions recorded in the data). The model learns to reduce random noise to meaningful motion paths through multiple data pairs. The dirt judgment model is a common classification model. By collecting data pairs, it determines whether to proceed to the next step or maintain the current step (similar to wiping again if not clean enough) for the current visual input (such as an image of a dirty toilet seat).

[0061] The wrist / hand components of a robotic arm are typically designed as grippers, but this invention relates to a replaceable, plug-and-play configuration that can be adapted to end devices with 0 / 1 signals, such as grippers (1 off, 0 on), electromagnets, and suction cups. The invention, by default, uses an electromagnetic adsorption device that is friendly to the ceramic surface of a toilet (gripper-type devices may scratch the surface).

[0062] ROS is short for Robot Operating System, with "2" indicating a second-generation operating system. It is an integrated control flow system and the main communication method used in this invention.

[0063] This invention uses ESP32 as the end effector chip to control functions such as the switching on and off of electromagnets and the opening and closing of grippers, and communicates with the main control unit via Bluetooth.

[0064] This invention includes several communication mechanisms: one is internal communication within the robotic arm, using the robotic arm's own CAN interface; the second is communication between the robotic arm and the main controller, via a CAN-to-USB interface; the third is communication between the microcontroller and the main controller, via Bluetooth signals generated by the microcontroller; and the fourth is communication between the main controller and the program, as well as communication between programs, via ROS2.

[0065] The specific objective of this invention is: 1. Eliminate secondary pollution and simplify maintenance. The cleaning pads are disposable to prevent dirt buildup; the electromagnets instantly attract / detach (or use the grippers) when ON / OFF, and users only need to replace the consumables.

[0066] 2. User-selectable end effector This invention provides a quick-release end effector kit that can be rapidly replaced with a gripper or an electromagnetic adsorption device. The electromagnetic adsorption device is a lower-cost design and more surface-friendly for toilets. Robot grippers are typically made of hard plastic or alloy, which may scratch furniture surfaces when grasping thin cleaning tools (such as scouring pads or cloths) due to the hardness of the gripper. The electromagnet design avoids this problem because its surface is very smooth. For cleaning tools, this invention uses a magnetic sheet attached to a disposable scouring pad, simplifying the materials. The robotic arm, including the end effector, does not directly contact the toilet; only the scouring pad comes into contact with the toilet. However, to cope with complex environments, the possibility of using grippers or other end effects is still retained. To accommodate multiple end effectors, the internal system program has been developed to support various devices controlled by 0 / 1 signals, allowing for rapid adaptation.

[0067] 3. Simplify positioning and improve adaptability It uses dual-camera vision and Diffusion model to directly generate the end-effector 6DoF trajectory and magnet status, eliminating the need for laser SLAM or depth cameras and avoiding interference from humid mirror environments on the sensors.

[0068] 4. Reduce size and lower deployment threshold The lightweight single arm and simple base can cover all areas of the toilet; installation only requires fixing the base, without taking up extra space.

[0069] 5. Reduce overall manufacturing and maintenance costs The simplified mechanical structure and sensor chain are compatible with commercially available disposable scouring pads, meeting the comprehensive needs of home users for cost, hygiene, and ease of use.

[0070] In another embodiment, the present invention uses AI to control a robotic arm (mounted on a movable chassis) to clean a household toilet. The toilet cleaning robot of the present invention consists of three parts: a perception layer, a decision control layer, and an execution layer. These three layers exchange data via a ROS2 bus to achieve a closed loop of "visual input → AI reasoning → cleaning action".

[0071] The specific node relationships in ROS2 are as follows: The ` / vision / bridge_node` subscribes to the two camera topics ` / cam_base / image_raw` and ` / cam_tip / image_raw`, and then feeds them into the AI ​​model after synchronization. / endpose_state is the state information transmitted back by the robotic arm at a fixed frequency, referring to the spatial attitude of the end effector, x, y, z, ow, ox, oy, oz. The ` / joint_state` function subscribes to the status information transmitted back from the three robotic arms at a fixed frequency, including the joint angles (` / joint_value`), joint velocities (` / joint_velocity`), and joint torques (` / joint_effort`). After synchronization, this information is sent to the main control unit. The ` / endpose_cmd` main control unit acquires the target space pose (instruction) output by the model, and performs filtering, interpolation, and judgment. The ` / magnet_state` main control unit subscribes to custom electromagnet state information. If the electromagnet is currently high (on), this channel will broadcast `true` at a fixed frequency; otherwise, it will broadcast `false`. The / magnet_cmd main control unit obtains the control commands for the electromagnet output by the model. After performing appropriate processing, such as preventing the same value from being sent multiple times consecutively (to avoid channel congestion, as the electromagnet's carrier chip is an ESP32 with limited processing capabilities), it sends the command to control the electromagnet to turn on and off to the electromagnetic unit.

[0072] Execution layer: Robotic arm unit: 6-joint lightweight servo arm with a stroke sufficient for full coverage cleaning of the toilet; Electromagnetic unit: 5V DC electromagnet (attractive force ≥10N), driven by ESP32 MOSFET (logic level PWM), on / off time <30ms; Low-requirement mobile chassis: Only provides XY translation and slight rotation, and can be positioned manually or by voice prompts, without autonomous navigation hardware.

[0073] The specific circuit connections include: 1. ESP32 driver circuit GPIO23 outputs PWM, which is then passed through the gate of an N-channel MOSFET (IRLZ44N); the source is grounded; the drain is connected in series with an electromagnet coil and then connected to 24V; a 5V→3.3V LDO regulator powers the ESP32; the Hall current detection module feeds back the coil current to ADC1_CH0; the Bluetooth RFCOMM uses the SPP protocol with a baud rate of 115 200bps.

[0074] 2. Camera power supply and data Two USB 3.0 ports connect to the main x86 control board; it provides 5V / 900mA, the image acquisition node uses MJPEG compression, the frame rate is 10Hz, and it occupies a bandwidth of approximately 15Mbps.

[0075] 3. Robotic arm control The joint servo uses a CAN bus for communication.

[0076] Electromagnet control logic: magnet_node only switches the magnet state when the predictions are consistent for 3 consecutive frames, to avoid false triggering.

[0077] Electromagnetic energization lasts for a maximum of 30 seconds; after this timeout, the power will automatically cut off and an alarm will be issued.

[0078] After the robot passes the self-inspection, it enters the cleaning phase, performing three layers of cleaning in sequence: First layer (exterior): Clean the outer surface of the toilet seat → Base camera detects remaining dirt in real time; If the test result is N (not clean) → repeat this layer; if it is P → execute S 6.3 to open the cover and perform subsequent actions.

[0079] Second layer (seat ring): Guided by vision, the robotic arm opens the seat ring, absorbs a new cleaning pad, wipes the outer and inner walls of the seat ring, and moves to the next layer after the cycle is completed and the seat ring is deemed clean.

[0080] Third layer (base): Clean the toilet base and under the seat; after cleaning, close the seat and toilet lid, flush in S9 mode and return to standby mode.

[0081] Each cleaning layer internally calls a "workflow to execute a specific cleaning task": 1. Model selection: Load the corresponding Diffusion sub-model weights (outer wall / inner wall / base) according to the task type; 2. Status information capture: Simultaneous capture of two-channel images + end-effector encoder posture; 3. Model inference: Iteratively generate action sequences {pose i, magnet i}; 4. Cleaning completion determination: based on the threshold of image difference and brightness histogram < ε; 5. If N (incomplete, negative) → return to step 2; if P (positive) → end and publish the event.

[0082] The present invention also includes: a toilet cleaning robot, comprising a replaceable end effector and a disposable magnetic cleaning pad; dual-camera visual perception; outputting end effector posture and electromagnet switching signals from a generative Diffusion model; and communicating with the execution layer via a ROS2 bus to complete cleaning.

[0083] The electromagnetic adsorption-desorption control method includes: predicting the electromagnet state signal; and continuous multi-frame consistency verification. A multi-layer toilet cleaning method based on a visual diffusion model sequentially cleans the outer surface, seat, and base, automatically replacing disposable cleaning pads at each layer. It includes thresholds for determining dirt and repeated wiping.

[0084] The lightweight mobile chassis and single-arm collaborative system only provide XY micro-displacement and rotation of the chassis and does not include SLAM / LiDAR devices. Instead, positioning and cleaning are completed through the collaboration of cameras and robotic arms.

[0085] ROS2-Bluetooth-ESP32 control link device: The ROS2 node encapsulates the magnet command as the / magnet / cmd topic; the BluetoothSerial SPP protocol is sent to the ESP32; the ESP32 controls the 24V electromagnet through MOSFET and sends the status back to / system / state.

[0086] The magnetic fixing structure of the disposable cleaning pad includes: a magnetic metal sheet embedded in the back of the cleaning pad; a flexible substrate (non-woven fabric / scouring pad); and a pressing force ≥N when it is attached to the electromagnet and can be naturally detached in the OFF state.

[0087] In terms of the device, the camera and the main controller are connected via USB, the robotic arm and the main controller are connected via CAN to USB (the robotic arm itself communicates using CAN signals), the electromagnet is connected to the ESP32 via DuPont wires, the ESP32 communicates with the main controller via Wi-Fi, and is powered by the main controller via micro-USB.

[0088] For the connection between the microcontroller and the electromagnet module: connect the "+" of the electromagnet to the "VIN" of the microcontroller, connect the "-" of the electromagnet to the "GND" of the microcontroller, and connect the "S" of the electromagnet to the "GPIO 13" of the microcontroller.

[0089] The robotic arm currently used is from the Piper series of Songling robots. It has 6 degrees of freedom (joints), and each joint has a specific range of rotation to prevent the robotic arm from self-collision or excessive rotation. The first joint (J1), the second joint (J2), the third joint (J3), the fourth joint (J4), the fifth joint (J5), and the sixth joint (J6) are as follows: J1:±154° J2: 0°~195° J3: -175°~0° J4: -102°~102° J5: -75°~75° J6:±120° J1 is the base joint of the robotic arm, which allows the robotic arm to rotate left and right in the horizontal direction. J2, J3, and J5 are all vertical bending joints, while J4 and J6 are rotation joints in any space.

[0090] In addition, each joint has a specific purpose and configuration, and therefore a specific joint speed limit, as follows: J1: 180° / s J2: 195° / s J3: 180° / s J4:225° / s J5:225° / s J6:225° / s.

[0091] One aspect of this invention involves an electromagnet. The robotic arm's kit doesn't support other end effectors (originally intended to be Songling's own gripper), but it was upgraded to the current electromagnet device to meet specific needs. The connection between the electromagnet device and the end effector is described below. A 3D-printed fastener secures the electromagnet and the robotic arm. Specifically, the installation involves fixing a 3D-printed plastic support to the exposed flange of the robotic arm using six screws. The entire function of this support is to connect the six screws to the flange and the four screws to the electromagnet fastener.

[0092] The installation method for electromagnets can be referenced from that of clamps, which should be relatively simple. The installation method for electromagnets is also the same.

[0093] The connection between the electromagnet and the microcontroller is also relatively simple: The electromagnet module has three wires that need to be connected: positive and negative wires and a signal wire. If you assemble it exactly according to the program, you only need to connect the signal wire to pin 27 of the ESP32 microcontroller, and the positive and negative wires to the VCC and GND ports of the ESP32. After connecting them, properly embed the microcontroller and the electromagnet module into the 3D printed support, and then fix the two components with screws and nuts.

[0094] The assembly logic for the camera is also very simple. 3D-printed support parts are installed on joints 6 and 1 of the robotic arm and fixed with screws and nuts to ensure that the cameras on the chest and wrist can reasonably see the object. The model of this invention allows for a certain degree of visual offset. For example, if the installation position is too high or too low, or slightly crooked, as long as the line of sight is not obstructed, the model of this invention can adapt.

[0095] This invention is not limited to the above-described embodiments. Any changes made to its shape or material composition, or any structural design using the methods provided by this invention, are considered variations of this invention and should be considered within the scope of protection of this invention.

Claims

1. A method for controlling a toilet cleaning robot, characterized in that, include: Visual information is collected synchronously by dual cameras, and data on the posture and joint status of the robotic arm's end effector are obtained. Based on visual input, a three-layer cleaning decision model is invoked, and the corresponding action generation model for the cleaning level is dynamically loaded. Based on image and end-effector posture data, the motion trajectory of the end-effector and electromagnet control commands are generated in real time through the corresponding trajectory generation model. After verification for several consecutive frames, the robotic arm is driven to perform the wiping action. The cleaning completion level is determined based on image difference and brightness threshold. If the standard is not met, the current layer cleaning cycle is maintained. Once the standard is met, the process switches to the next cleaning layer.

2. The toilet cleaning robot control method according to claim 1, characterized in that, The three-layer cleaning decision model divides the action of cleaning the toilet into several cleaning action sequences. Each cleaning action sequence is trained using collected training data, allowing the model to determine whether to repeat the cleaning, perform the previous action, or proceed to the next action label under the current visual input. The training data is labeled with numbers corresponding to the cleaning action sequences, and several data augmentation methods are randomly applied to the training data.

3. A toilet cleaning robot control method according to claim 1 or 2, characterized in that, The input data for the trajectory generation model includes: A four-channel tensor composed of synchronized images from the current and previous cameras. A 20-dimensional vector is formed by combining the current and previous moments' end-effector pose and rotational six-dimensional representations with the gripper state. The four-channel tensor and the twenty-dimensional vector are fused together by the observation encoder and then input.

4. The toilet cleaning robot control method according to claim 3, characterized in that, The trajectory generation model generates the motion trajectory of the end effector and the electromagnet control commands in real time. Specifically, it includes: the motion prediction model based on the Transformer architecture outputs the current state noise, generates 16 future actions by iterating through 10 steps, executes the first 8 actions, and uses the last 8 actions as a timing buffer.

5. A toilet cleaning robot control method according to claim 2 or 4, characterized in that, The method of synchronously acquiring visual information through dual cameras includes synchronously acquiring visual information, identifying the toilet mask area in the camera image through a visual model, calculating the average depth value of the mask area by combining the depth camera data, and prompting the user to adjust the robot position if the average depth value is greater than the working radius of the robotic arm.

6. The toilet cleaning robot control method according to claim 2, characterized in that, The trajectory generation model includes several independent sub-models, each sub-model corresponding to a cleaning action sequence, with a consistent architecture. The training data is allocated according to the task complexity, specifically including low-complexity tasks using more than 100 teleoperation teaching data and a training period of less than 24 hours, and high-complexity tasks using more than 1000 teleoperation teaching data and a training period of 3-7 days. The electromagnet control commands include three consecutive frames of predicted state consistent execution commands, and a determination of whether the duration of a single power-on exceeds 30 seconds, with automatic power-off alarm in case of timeout.

7. The toilet cleaning robot control method according to claim 6, characterized in that, The step of determining the cleaning completion level based on image difference and brightness threshold includes: calculating the difference matrix using partitioned block sampling, extracting suspected dirty areas, and statistically analyzing the brightness histogram of suspected dirty areas. If the standard deviation or mean does not meet the requirements, it is determined that the cleaning has not met the standards. Specifically, the step of calculating the difference matrix using partitioned block sampling includes: dividing the cleaning area into a grid, starting the difference calculation for the grid blocks covered by the robotic arm's end effector trajectory, and dynamically ignoring reflective areas.

8. A toilet cleaning robot control system, employing the toilet cleaning robot control method according to any one of claims 1-7, characterized in that, include: The robotic arm includes several rotary joints, and the dual-camera vision acquisition module is deployed on the base and end of the robotic arm. The end of the robotic arm is equipped with a replaceable actuator module, and the base is connected to a mobile chassis module.

9. A toilet cleaning robot control system according to claim 8, characterized in that, The actuator module includes a gripper and an electromagnet module. The electromagnet module has an electromagnet control unit and an electromagnet support. The electromagnet support connects the robotic arm and the electromagnet control unit. The electromagnet module adsorbs disposable magnetic cleaning sheets.

10. A toilet cleaning robot control system according to claim 9, characterized in that, The rotary joint includes: a first joint (1) located on the base, which controls the main arm (8) to rotate left and right in the horizontal direction; The second joint (2) connecting the base and the main arm (8) controls the bending of the main arm (8) in the vertical direction; The third joint (3) connecting the main arm (8) and the secondary arm (9) controls the vertical bending of the secondary arm (9); The fourth joint (4) and the fifth joint (5) of the connecting functional block (10) are rotated in any direction, and the fifth joint (5) controls the bending in the vertical direction; Connect the function block (10) to the sixth joint (6) at the end, and control the end to rotate in any direction.

Images