A toilet uncovering device and system based on machine vision and sucker hook cooperation

Abstract

The application discloses a toilet cover opening device and system based on machine vision and sucker hook claw cooperation, and belongs to the technical field of machine vision robot operation control. The application comprises a visual perception module, a road point and normal generation module, an end pose and tool compensation module, a mechanical arm movement module, a sucker air path control module, a hook claw steering engine control module and a state machine decision module. The visual perception module detects a toilet cover, outputs an edge mask and a local plane point set, and sorts road points. The road point and normal generation module selects road points in sequence, fits an outer normal of a cover surface to generate a pre-approaching point. The end pose and tool compensation module calculates a flange target contact position. The mechanical arm movement module sequentially performs pre-approaching contact, two-stage normal lifting, hook claw intervention and load bearing, and completes side shifting and cover opening. The state machine decision module sets a Boolean type safety interlocking permission condition, and retreats and switches candidate road points if the condition is not met.

Description

Technical Field

[0001] This invention relates to the field of machine vision robot operation control technology, specifically a toilet lid opening device and system based on the collaboration of machine vision and suction cup claw. Background Technology

[0002] The use of a robotic arm to open the toilet lid is a prerequisite for automated toilet cleaning. The surface of the toilet lid has unevenness, and the area that can be suctioned is limited. The robot needs to rely on vision to identify the appropriate contact point and then use the end effector to complete the opening action. If the point is not selected accurately, problems such as air leakage from the suction cup and failure of suction may occur, resulting in failure of the opening task. Most existing robotic toilet lid opening solutions rely on suction cups to lift and open the lid, which is highly dependent on the negative pressure of the suction cups; they lack a complete suction point screening and sorting system, which can easily cause suction cup failure during lifting and load transfer; and they have insufficient fault tolerance. Therefore, there is an urgent need for a toilet lid opening system that uses a suction cup and a claw in coordination under machine vision guidance. By visually selecting the suction contact point and using a waypoint switching retry mechanism, the reliance on suction cup negative pressure can be reduced, thereby improving the reliability and fault tolerance of the cleaning robot's toilet lid opening operation. Summary of the Invention

[0003] The purpose of this invention is to provide a toilet lid opening device and system based on machine vision and suction cup claw collaboration, so as to solve the problems mentioned in the prior art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a toilet lid opening system based on machine vision and suction cup / claw collaboration. The toilet lid opening system includes a vision perception module, a waypoint and normal generation module, an end-effector pose and tool compensation module, a robotic arm motion module, a suction cup air path control module, a claw servo control module, and a state machine decision module. It is suitable for cleaning robots equipped with vision sensors and robotic arms, with the target object being a toilet lid. The toilet lid opening system uniformly uses the robot base coordinate system B to represent position and orientation, and the flange target coordinate system T. k This represents the end effector posture corresponding to the k-th candidate waypoint. All vectors are transformed to the robot base coordinate system B before being substituted into the formula. The flange target coordinate system is a coordinate system with the robot flange reference point as the origin, used to describe the target posture of the end effector. The visual perception module performs visual detection on the toilet area and outputs an edge mask, a local planar point set, and a candidate waypoint set; providing geometric basis and candidate positions for subsequent adsorption operations. The visual perception module includes a feature extraction unit and a waypoint sorting unit; The feature extraction unit performs visual detection on the toilet area, captures an image of the toilet seat, and outputs an edge mask, a local planar point set, and a candidate waypoint set. The edge mask is a binary pixel region in the toilet seat image that marks the area along the toilet seat edge, used to limit the search range of the suction point. The local planar point set is a set of three-dimensional points located on the toilet seat surface within the area near the toilet seat edge, used to fit the local plane of the lid surface of the robot suction cup adsorption area. The candidate waypoint set consists of different three-dimensional position points detected at intervals along the toilet seat edge, with each three-dimensional position point serving as a candidate waypoint for the suction cup to contact. The waypoint sorting unit sorts candidate waypoints according to local flatness. Local flatness represents the degree of deviation of toilet seat surface points in the neighborhood of a candidate waypoint relative to the fitted local plane. The neighborhood is a set of local plane points within a preset radius centered on the candidate waypoint. The preset radius is set to ensure that enough points are included for plane fitting without introducing points outside the lid edge. Toilet seat surface points are three-dimensional points located on the toilet seat surface from the local plane point set. The fitted local plane is a plane representing the local shape of the toilet seat surface obtained by fitting the toilet seat surface points. The degree of deviation is the statistical value of the distance from each toilet seat surface point to the fitted local plane. The smaller the distance, the smaller the degree of deviation and the higher the flatness, resulting in a more reliable seal after the suction cup contacts. Candidate waypoints are sorted from high to low local flatness.

[0005] The waypoint and normal generation module selects candidate waypoints sequentially from the candidate waypoint set in order, and obtains the out-of-cover normal by fitting the local planar point set of the neighborhood of the current candidate waypoint; at the same time, it generates a pre-approach point along the outer side of the out-of-cover normal as an intermediate transition position for the robotic arm to approach the edge of the toilet seat. When the current candidate waypoint fails, it automatically switches to the next candidate waypoint. The waypoint and normal generation module includes a waypoint selection unit, a normal estimation unit, and a pre-approach point generation unit; The waypoint selection unit selects candidate waypoints sequentially from the candidate waypoints in order of sorting. The index of the currently selected candidate waypoint is denoted as k, and k starts from 1 and increments sequentially, k=1,2,…,K, where K is the total number of valid candidate waypoints. If the current candidate waypoint fails in subsequent operations, it is marked as failed and the next candidate waypoint with index k+1 is selected. The normal estimation unit takes a local planar point set in the neighborhood of the current candidate waypoint and obtains the out-of-cover normal n through planar fitting. k The outer normal of the lid is a unit vector perpendicular to the toilet lid surface and pointing from the toilet lid surface to free space. Free space refers to the open space on the side of the toilet lid surface away from the toilet seat, which is unobstructed and allows the suction cup to approach and lift. The pre-approach point generation unit is located outside the cover surface along the normal direction, at a pre-approach distance d from the candidate waypoint. preThe location is designated as the pre-approach point, which is the intermediate transition point where the robot arm will reach the toilet seat and then cut in along the normal straight line, so that the toilet seat is lifted towards one side. The pre-approach distance d is specified in the original text. pre It is a preset fixed distance value, which is set based on the axial length of the suction cup tool and the safety margin of the robotic arm to determine the safe obstacle avoidance distance.

[0006] The end-effector pose and tool compensation module determines the end-effector pose based on the outer normal of the cover surface, making the tool axis collinear with the outer normal of the cover surface in opposite directions, constructing three coordinate axes of the flange target coordinate system, calculating the tool offset vector through the tool center point calibration, and inversely deducing the flange contact position. The end-effector pose and tool compensation module includes a pose determination unit, a coordinate system construction unit, and a tool compensation unit; The attitude determination unit determines the attitude of the end tool based on the outward normal of the toilet seat surface, ensuring that the suction cup contact surface is directly facing the toilet seat surface; tool axis z T,k The positive direction is fixed as from the flange origin to the suction cup contact surface, and the outer normal n of the cover surface. k The positive direction is fixed as from the cover surface to free space, and the two directions are opposite. Therefore, when the suction cup is facing the cover surface, the tool axis and the outer normal of the cover surface satisfy the relationship: z T,k =-n k In the formula, z T,k Let n be the unit direction vector of the flange target coordinate system tool axis under B for the k-th candidate waypoint. k Let be the out-of-cover unit normal vector at the k-th candidate waypoint; The coordinate system construction unit selects a reference vector that is not parallel to the tool axis, and constructs a right-handed orthogonal coordinate system through the cross product of vectors to obtain x. T,k y T,k , and z T,k The three coordinate axes that together constitute the target coordinate system of the flange, where x T,k Let x be the unit direction vector of the X-axis in the target coordinate system of the flange, and the formula is: T,k =(a B ×z T,k ) / |(a B ×z T,k )|;y T,k Let y be the unit direction vector of the Y-axis in the target coordinate system of the flange, and the formula is: T,k =z T,k ×x T,k In the formula, a B It is the reference unit vector used to construct the tool's pose, expressed in B, and is not related to z. T,k parallel; The tool compensation unit calculates the tool offset vector t from the origin of the flange target coordinate system to the suction cup contact center point. FC,k :tFC,k =Lz T,k +o x x T,k +o y y T,k In the formula, L is the axial distance from the origin of the flange target coordinate system along the tool axis to the suction cup contact surface; L > 0. x o y The center point of the suction cup contact is at x T,k y T,k The offset in the plane relative to the flange axis, where the flange axis is the centerline extending from the origin of the flange target coordinate system along the tool axis direction; then, the flange contact position p corresponding to the k-th candidate waypoint is calculated. F,k c :p F,k c =p k -t FC,k In the formula, p F,k c For suction cup contact p k Flange target position at time, p k Let be the three-dimensional position vector of the k-th visual contact candidate point under B.

[0007] The robotic arm motion module sequentially completes pre-approach, normal contact, first stage normal lifting, second stage normal lifting, side-moving opening and return, so that the suction cup smoothly contacts the edge of the toilet seat and the hook pulls the toilet seat ring component to rotate around the hinge to the preset opening angle. The robotic arm motion module includes a pre-approach contact unit, a normal lifting unit, and a side-moving cover opening unit; The pre-approach contact unit executes the robotic arm movement according to the target pose. The robotic arm first moves to the pre-approach point, and then moves linearly at a limited speed along the outer normal of the cover surface to the flange contact position p. F,k c Afterwards, the adsorption is confirmed, and the restricted speed is a preset value lower than the normal movement speed of the robotic arm. This is to prevent the suction cup from impacting the edge of the toilet seat and to ensure that the suction cup makes smooth contact with the toilet seat edge area. The formula for the position of the pre-approach point is: p F,k pre =p F,k c +d pre n k In the formula, p F,k pre This is the pre-approach point before the suction cup contacts the edge of the toilet seat; After adsorption confirmation, the normal lifting unit performs the first stage of normal lifting, raising the toilet seat a first distance along the outer normal direction away from the toilet seat surface, creating an initial gap between the toilet seat and the toilet seat body for the hook to enter. After the hook bears the load and releases pressure, the second stage of normal lifting is performed, continuing to raise the seat a second distance along the same normal direction. The flange position at the end of the two stages of lifting is p. F,k 1 p F,k 2 ;where p F,k 1 =p F,k c +d1n k p F,k 2 =p F,k c +(d1+d2)n k In the formula, p F,k 1 p F,k 2 These are the flange positions at the end of the first and second normal lifting sections, respectively. d1 and d2 are the first and second distances, respectively. d1 is determined based on the thickness of the hook and the gap requirement of the cover edge, and d2 is determined by the safety margin for the suction cup to detach from the toilet seat surface. Preferably, d1 is 20mm and d2 is 10mm. The side-moving lid-opening unit performs side-moving lid opening after two stages of normal lifting, translating along a preset direction in the robot base coordinate system. This causes the chuck to pull the toilet seat ring component to rotate around the hinge. After the lid-opening termination condition is met, the robotic arm returns to the default joint pose. The target flange position at the end of the side-moving lid opening is p. F,k 3 :p F,k 3 =p F,k 2 +d s e s In the formula, p F,k 3 It is the target flange position at the end of the side-sliding cover opening, e s The preset unit direction for the side-opening lid is expressed in B, d. s For along e s lateral displacement, d s >0; preferred e s Take the negative X-axis direction of the base coordinate system, d s The opening angle is set to 250mm. The opening termination condition is that the toilet seat ring component rotates to the preset opening angle. The opening angle is set according to the requirements that the toilet seat ring should not obstruct the operation of the cleaning robot after it is fully opened, and should not collide with the water tank or obstacles behind it.

[0008] The suction cup air path control module manages the switching between the suction channel and the exhaust channel. After the suction cup makes contact, the suction is turned on and the vacuum is confirmed by the negative pressure sensor to determine whether the adsorption is effective. If no effective adsorption is formed, the suction is retracted and the candidate path point is switched. When effective adsorption is formed, the suction is turned off and the exhaust is turned on to actively depressurize, so that the load is quickly transferred from the suction cup to the hook. The suction cup air circuit control module includes an air circuit switching unit and a pressure relief control unit; The air path switching unit manages the switching between the intake and exhaust channels. The intake channel generates negative pressure, allowing the suction cup to adhere to the toilet seat surface. The exhaust channel actively introduces air to break the negative pressure. When the first stage of normal lifting begins, the intake channel is opened and the exhaust channel is closed. The negative pressure sensor is used to perform the first vacuum confirmation. The negative pressure sensor measures the air pressure inside the suction cup in real time. The absolute value of the measured negative pressure is compared with a preset first negative pressure threshold. If the first negative pressure threshold is reached or exceeded, the first vacuum confirmation is passed. Otherwise, the suction cup retracts, and the next candidate waypoint is switched. The first negative pressure threshold is determined based on the minimum negative pressure value required for the maximum load required for the first stage of normal lifting. After the first stage of normal lifting, the negative pressure sensor is read for a second vacuum confirmation. The measured absolute value of negative pressure is compared with the preset second negative pressure threshold. If it reaches or exceeds the second negative pressure threshold, it is determined to be effective adsorption and the second vacuum confirmation is passed. Otherwise, it is determined that no effective adsorption has been formed. The second negative pressure threshold is determined according to the minimum negative pressure value required for the maximum load required for the second stage of normal lifting. When the pressure relief control unit determines that the adsorption is effective, it executes the hook grabbing action. After the hook is in position, it closes the air intake channel and opens the exhaust channel to relieve pressure, so that the load is quickly transferred from the suction cup to the hook. After completion, it closes the air intake and exhaust channels. The hook grabbing action refers to the action of the servo motor moving from the storage position to the grabbing position, so that the hook enters the gap formed by the toilet seat edge and abuts against the edge of the seat. The hook being in position refers to the hook reaching the grabbing position and entering below the edge of the seat. In the pressure relief control unit, the intake channel state is recorded as U. suc U suc A value of 0 indicates that the connection is closed, U. suc A value of 1 indicates the exhaust passage is open; the exhaust passage status is recorded as U. vent U vent A value of 0 indicates that the connection is closed, U. vent A value of 1 indicates the airway is open. Before the adsorption is established and the hook reaches its position, the airflow status is (U). suc U vent )=(1,0); After the load is transferred from the suction cup to the hook, the air path state switches to (U suc U vent = (0,1); After the toilet lid is opened and the pressure is released, the gas path status is (U) = (0,1); suc U vent= (0, 0).

[0009] The hook servo control module drives the hook to switch between the storage position and the hooking position. During the approach, contact and first normal lifting of the suction cup, the hook is kept in storage. After the initial gap is formed, it switches to the hooking position so that the hook enters the gap and takes over the subsequent opening load from the suction cup. During the return phase, the hook is retracted to the storage position. The hook servo control module includes a hook drive unit and an attitude switching unit; The hook drive unit drives the hook to switch between a retracted position and a hooking position. The hook has at least two connecting holes spaced along its longitudinal direction to adjust the length of the working end of the hook relative to the servo shaft. The working end of the hook refers to the bent end of the hook that extends under the toilet seat rim, directly contacts the toilet seat rim, and bears the hooking load. The retracted position is the servo angle position where the hook is retracted and does not obstruct the suction cup from approaching the toilet seat rim. The hooking position is the servo angle position where the hook is extended and the working end enters under the rim of the toilet seat. During the first stage of normal lifting of the suction cup, the pawl remains in the storage position. After the initial gap is formed, the servo motor moves from the storage position to the hooking position, so that the pawl enters the formed gap and abuts against the edge of the cover, taking over the load of subsequent opening of the cover from the suction cup. The value of the servo motor position corresponding to the hooking position depends on the servo motor model and zero position. In the return phase, the servo motor is driven from the hooking position to the storage position. The return phase refers to the final stage where the robot arm returns to the preset joint posture after the lateral opening of the cover reaches the opening termination condition.

[0010] The state machine decision module uses a hook to hook the permission condition and the side-moving lid opening permission condition for safety interlocking. When the permission condition is met, the system proceeds to the next stage; when it is not met, failure processing is performed, including retracting along the outer normal of the lid and switching to the next candidate waypoint, and returning the toilet lid to its original path. When all K candidate waypoints fail, the system exits the task and issues an alarm. The state machine decision module includes an execution permission unit and a failure handling unit; Upon successful completion of the first and second vacuum verifications, the execution authorization unit allows the grappling hook to enter the hooking position. The hooking authorization condition is C. hook,k =A k Q1Q2; in the formula, A in the formula k Q1 and Q2 are Boolean values ​​that can take the value 0 or 1. When they are multiplied directly, the product is 1 only if all three are 1; if any one of them is 0, the product is 0. kQ1 is the executable flag for the k-th candidate waypoint. It is 1 if the flange target pose corresponding to the current candidate waypoint is within the robot arm's workspace, and 0 otherwise. Q2 is the first vacuum confirmation result; it is 1 if the negative pressure reaches the threshold, and 0 otherwise. Q3 is the second vacuum confirmation result; it is 1 if the negative pressure reaches the threshold, and 0 otherwise. hook,k A value of 0 disallows the hook from entering the hooking position, while a value of 1 allows the hook to enter the hooking position. After the hook enters the hooking position, and after confirming that the air intake channel is closed and the exhaust channel is open, side-sliding cap opening is permitted. The permitted condition for side-sliding cap opening is C. side,k =C hook,k H(1-U suc )U vent In the formula, C hook,k H, (1-U) suc ), U vent The four factors are all either 0 or 1. Multiplying them directly results in a product of 1 only if all four factors are 1; otherwise, the product is 0. (C) side,k To allow the k-th waypoint to perform the side-shift opening, H represents the confirmation result of the grappling hook reaching the hooking position; it is 1 when the servo is in position, and 0 otherwise. C side,k When the value is 0, the lid cannot be opened by sliding it to the side; when the value is 1, the lid can be opened by sliding it to the side. The failure handling unit is in C hook,k If C is 0, the current candidate waypoint is deemed a failure. The intake channel is closed, the exhaust channel is opened, and the system retreats along the outer normal of the cover surface to the pre-approach point. Then, the next candidate waypoint with index k+1 is selected. side,k When the value is 0, the current candidate waypoint is considered to have failed, depressurization and side-moving opening of the toilet seat are prohibited, and the toilet seat is returned to its original path. When all K candidate waypoints have failed, the system exits the task and issues an alarm.

[0011] A toilet lid opening device based on machine vision and suction cup / claw collaboration, the toilet lid opening device includes an electric vacuum suction cup, a servo motor, a servo motor swing arm, a claw, a magnetic metal block, and a mounting structure; the electric vacuum suction cup, servo motor, servo motor swing arm, claw, and magnetic metal block share a common mounting structure as a geometric reference, so that the center of the suction cup, the axis of the servo motor, and the claw maintain a calibrable and fixed relationship; The mounting structure is the support base of the device. The mounting structure has a suction cup mounting surface, a servo motor mounting surface and a metal block mounting surface, which are used to fix the electric vacuum suction cup, the servo motor and the magnetic metal block, respectively.

[0012] The electric vacuum suction cup is fixed to the suction cup mounting surface of the mounting structure, with the suction cup contact surface facing downwards, used to establish a flexible negative pressure connection with the toilet seat edge and complete the initial lifting; the electric vacuum suction cup and the mounting structure are fastened together by screws, so that the position of the suction cup center relative to the mounting structure is fixed. The magnetic metal block is fixed to the metal block mounting surface of the mounting structure and is located at the top of the mounting structure. It is used to cooperate with the electromagnetic holding parts on the robot wrist adapter and the mobile carrier tool position to realize the transfer of the opening end between the robotic arm and the tool parking position. The magnetic metal block is fastened to the mounting structure by screws or embedded fixation. The servo motor is fixed to the servo motor mounting surface of the mounting structure and is located on the side of the mounting structure. The servo motor output shaft faces outward and is used to drive the servo motor swing arm to rotate. The servo motor and the mounting structure are fastened together by at least two screws to fix the position of the servo motor axis relative to the mounting structure. One end of the servo arm is connected to the output shaft of the servo motor and rotates synchronously with the output shaft. The other end is connected to the pawl and is used to transmit the rotational motion of the servo motor to the pawl. The servo arm and the output shaft of the servo motor are fixed together by a shaft hole and a locking screw, so that there is no relative rotation between the servo arm and the output shaft. The hook is connected to the other end of the servo arm. At least two connection holes are provided on the hook along the longitudinal direction of the hook. By selecting different connection holes to connect with the servo arm, the effective length of the working end of the hook relative to the servo shaft can be adjusted. The hook and the servo arm are fastened together by bolts passing through the connection holes. The hook switches between the storage position and the hooking position with the servo arm. The electric vacuum suction cup, servo motor, and magnetic metal block are all directly fixed to the mounting structure. The servo motor swing arm is indirectly mounted to the mounting structure via the servo motor, and the pawl is indirectly mounted to the mounting structure via the servo motor swing arm.

[0013] Compared with the prior art, the beneficial effects of the present invention are: 1. Based on visual local flatness, filter, sort, and switch adsorption waypoints: Use the local plane point set of the toilet seat to evaluate the degree of plane deviation of each candidate waypoint, and give priority to the adsorption position with better sealing conditions; when the adsorption of a single waypoint fails, automatically retreat and switch to the next candidate waypoint; when all waypoints fail, alarm and exit, improving the fault tolerance of the adsorption process.

[0014] 2. Achieve step-by-step load transfer between suction cup and hook: The first stage of suction cup lifting creates a gap along the toilet seat. After the hook extends into the gap, it actively releases pressure through the air passage, transferring the load of opening the lid from the suction cup to the hook. Subsequent lid opening actions do not completely rely on the negative pressure of the suction cup, reducing the risk of task failure caused by air leakage from the suction cup.

[0015] 3. Set up safety interlocks and corresponding failure handling logic: Set multi-factor Boolean judgment conditions for hook hooking and side-moving cover opening respectively. Only when the visual pose, vacuum negative pressure, hook in place and air circuit status are all satisfied can the next operation stage be allowed. When the link is abnormal, execute the retraction, return and waypoint switching actions to avoid the robot arm from malfunctioning and ensure the safety of the whole machine operation. Attached Figure Description

[0016] Figure 1 This is a schematic diagram illustrating the process of applying the present invention to a toilet lid opening system based on machine vision and suction cup claw collaboration; Figure 2 This is a schematic diagram of the structure of a toilet lid opening device based on machine vision and suction cup claw collaboration. Figure 3 This is a schematic diagram showing the hook in the storage position when the present invention is applied to a toilet lid opening device based on machine vision and suction cup hook collaboration. Figure 4 This is a schematic diagram showing the hook in the hooking position when the present invention is applied to a toilet lid opening device based on machine vision and suction cup hook collaboration. Figure 5 This is a schematic diagram showing the application of the present invention in a toilet lid opening device based on machine vision and suction cup claw collaboration, where the suction cup completes the first stage of normal lifting and an initial gap is formed between the toilet lid edge and the seat. Figure 6 This invention is applied to a toilet lid opening device based on machine vision and suction cup claw collaboration. The claw moves from the storage position to the hooking position, enters the gap between the lid and the lid edge, and abuts against the lid edge. Figure 7 This is a schematic diagram illustrating the application of the present invention in a toilet lid opening device based on machine vision and suction cup / claw collaboration, where the load is transferred from the suction cup to the claw and the second stage of normal lifting is performed. Figure 8 This is a schematic diagram illustrating the application of the present invention in a toilet lid opening device based on machine vision and suction cup hook collaboration, wherein the hook pulls the toilet lid ring component to rotate around the hinge to the open state.

[0017] The markings in the diagram are: 1-Electric vacuum chuck, 2-Servo motor, 3-Servo motor swing arm, 4-Claw, 5-Magnetic metal block, 6-Mounting structure, 7-Metal mounting surface, 8-Chuck mounting surface, 9-Servo motor mounting surface. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example: Figures 1-8 As shown, this invention provides a technical solution using a cleaning robot equipped with a depth camera and a 6-axis robotic arm as a carrier. Specifically, it is a toilet lid opening system based on machine vision and a suction cup gripper. The toilet lid opening system includes a vision perception module, a waypoint and normal generation module, an end-effector pose and tool compensation module, a robotic arm motion module, a suction cup air path control module, a gripper servo control module, and a state machine decision module. It is suitable for cleaning robots equipped with vision sensors and robotic arms, with the target object being a toilet lid. The toilet lid opening system uniformly uses the robot base coordinate system B to represent position and orientation, and the flange target coordinate system T. k This represents the end effector posture corresponding to the k-th candidate waypoint. All vectors are transformed to the robot base coordinate system B before being substituted into the formula. The flange target coordinate system is a coordinate system with the robot flange reference point as the origin, used to describe the target posture of the end effector. The visual perception module performs visual detection on the toilet area and outputs an edge mask, a local planar point set, and a candidate waypoint set; providing geometric basis and candidate positions for subsequent adsorption operations. The visual perception module includes a feature extraction unit and a waypoint sorting unit; The feature extraction unit performs visual detection on the toilet area, captures an image of the toilet seat, and outputs an edge mask, a local planar point set, and a candidate waypoint set. The edge mask is a binary pixel region in the toilet seat image that marks the area along the toilet seat edge, used to limit the search range of the suction point. The local planar point set is a set of three-dimensional points located on the toilet seat surface within the area near the toilet seat edge, used to fit the local plane of the lid surface of the robot suction cup adsorption area. The candidate waypoint set consists of different three-dimensional position points detected at intervals along the toilet seat edge, with each three-dimensional position point serving as a candidate waypoint for the suction cup to contact. The waypoint sorting unit sorts candidate waypoints according to local flatness. Local flatness represents the degree of deviation of toilet seat surface points in the neighborhood of a candidate waypoint relative to the fitted local plane. The neighborhood is a set of local plane points within a preset radius centered on the candidate waypoint. The preset radius is set to ensure that enough points are included for plane fitting without introducing points outside the lid edge. Toilet seat surface points are three-dimensional points located on the toilet seat surface from the local plane point set. The fitted local plane is a plane representing the local shape of the toilet seat surface obtained by fitting the toilet seat surface points. The degree of deviation is the statistical value of the distance from each toilet seat surface point to the fitted local plane. The smaller the distance, the smaller the degree of deviation and the higher the flatness, resulting in a more reliable seal after the suction cup contacts. Candidate waypoints are sorted from high to low local flatness.

[0020] The waypoint and normal generation module selects candidate waypoints sequentially from the candidate waypoint set in order, and obtains the out-of-cover normal by fitting the local planar point set of the neighborhood of the current candidate waypoint; at the same time, it generates a pre-approach point along the outer side of the out-of-cover normal as an intermediate transition position for the robotic arm to approach the edge of the toilet seat. When the current candidate waypoint fails, it automatically switches to the next candidate waypoint. The waypoint and normal generation module includes a waypoint selection unit, a normal estimation unit, and a pre-approach point generation unit. The waypoint selection unit selects candidate waypoints sequentially from the candidate waypoints in order of sorting. The index of the currently selected candidate waypoint is denoted as k, and k starts from 1 and increments sequentially, k=1,2,…,K, where K is the total number of valid candidate waypoints. If the current candidate waypoint fails in subsequent operations, it is marked as failed and the next candidate waypoint with index k+1 is selected. The normal estimation unit takes a local planar point set in the neighborhood of the current candidate waypoint and obtains the out-of-cover normal n through planar fitting. k The outer normal of the lid is a unit vector perpendicular to the toilet lid surface and pointing from the toilet lid surface to free space. Free space refers to the open space on the side of the toilet lid surface away from the toilet seat, which is unobstructed and allows the suction cup to approach and lift. The pre-approach point generation unit is located outside the cover surface along the normal direction, at a pre-approach distance d from the candidate waypoint. pre The location is designated as the pre-approach point, which is the intermediate transition point where the robot arm will reach the toilet seat and then cut in along the normal straight line, so that the toilet seat is lifted towards one side. The pre-approach distance d is specified in the original text. pre It is a preset fixed distance value, which is set based on the axial length of the suction cup tool and the safety margin of the robotic arm to determine the safe obstacle avoidance distance.

[0021] The end-effector pose and tool compensation module determines the end-effector pose based on the outer normal of the cover surface, making the tool axis collinear with the outer normal of the cover surface in opposite directions, constructing three coordinate axes of the flange target coordinate system, calculating the tool offset vector through the tool center point calibration, and inversely deducing the flange contact position. The end-effector pose and tool compensation module includes a pose determination unit, a coordinate system construction unit, and a tool compensation unit; The attitude determination unit determines the attitude of the end tool based on the outward normal of the toilet seat surface, ensuring that the suction cup contact surface is directly facing the toilet seat surface; tool axis z T,k The positive direction is fixed as from the flange origin to the suction cup contact surface, and the outer normal n of the cover surface. k The positive direction is fixed as from the cover surface to free space, and the two directions are opposite. Therefore, when the suction cup is facing the cover surface, the tool axis and the outer normal of the cover surface satisfy the relationship: z T,k =-n k In the formula, z T,k Let n be the unit direction vector of the flange target coordinate system tool axis under B for the k-th candidate waypoint. k Let be the out-of-cover unit normal vector at the k-th candidate waypoint; The coordinate system construction unit selects a reference vector that is not parallel to the tool axis, and constructs a right-handed orthogonal coordinate system through the cross product of vectors to obtain x. T,k y T,k , and z T,k The three coordinate axes that together constitute the target coordinate system of the flange, where x T,k Let x be the unit direction vector of the X-axis in the target coordinate system of the flange, and the formula is: T,k =(a B ×z T,k ) / |(a B ×z T,k )|;y T,k Let y be the unit direction vector of the Y-axis in the target coordinate system of the flange, and the formula is: T,k =z T,k ×x T,k In the formula, a B It is the reference unit vector used to construct the tool's pose, expressed in B, and is not related to z. T,k parallel; The tool compensation unit calculates the tool offset vector t from the origin of the flange target coordinate system to the suction cup contact center point. FC,k :t FC,k =Lz T,k +o x x T,k +o y y T,k In the formula, L is the axial distance from the origin of the flange target coordinate system along the tool axis to the suction cup contact surface; L > 0. x oy The center point of the suction cup contact is at x T,k y T,k The offset in the plane relative to the flange axis, where the flange axis is the centerline extending from the origin of the flange target coordinate system along the tool axis direction; then, the flange contact position p corresponding to the k-th candidate waypoint is calculated. F,k c :p F,k c =p k -t FC,k In the formula, p F,k c For suction cup contact p k Flange target position at time, p k Let be the three-dimensional position vector of the k-th visual contact candidate point under B.

[0022] Example 1: Using a cleaning robot as a carrier, visual perception and end-effector pose calculation are performed on a standard toilet seat. The preset radius of the candidate waypoint neighborhood is 15mm, and the total number of effective candidate waypoints K=5; the pre-approach distance d pre =50mm, tool axial distance L=80mm, suction cup contact center offset relative to flange axis o x =2mm, o y =-1mm, reference unit vector a B Take the positive Z-axis direction of the base coordinate system as (0,0,1); The visual perception module captures a depth image of the toilet area and outputs a set of local plane points along the edge mask and the toilet seat surface. Five candidate waypoints are detected at intervals along the edge. For each candidate waypoint, a local plane is fitted within a 15mm radius neighborhood. The average distance from each point to the plane is calculated as the deviation. The deviations of the five waypoints are 0.4mm, 0.6mm, 0.3mm, 0.8mm, and 0.5mm, respectively. They are sorted from highest to lowest flatness (i.e., from smallest to largest deviation). The waypoint with a deviation of 0.3mm is selected as the first candidate waypoint (k=1), and its 3D position in the base coordinate system B is p1=(300,50,200)mm. The outward normal to the toilet seat surface is obtained by fitting the neighborhood point set of the current waypoint, n1=(0.2,0,0.98), which is normalized and points away from the free space of the toilet seat. The tool axis z... T,1 =-n1=(-0.2,0,-0.98), so that the suction cup contact surface faces the cover surface; Construct the target coordinate system for the flange, first calculate the cross product a. B ×z T,1 =(0,0,1)×(-0.2,0,-0.98)=(0,-0.2,0), modulus 0.2, normalized to get x T,1 =(0,-1,0), then y T,1=z T,1 ×x T,1 Calculate y T,1 =(-0.98,0,0.2), the three axes together form a right-handed orthogonal coordinate system, and the calculation tool is the bias vector t. FC,1 =80×(-0.2,0,-0.98)+2×(0,-1,0)+(-1)×(-0.98,0,0.2)=(-16,0,-78.4)+(0,-2,0)+(0.98,0,-0.2)=(-15.02,-2,-78.6)mm, reverse the flange contact position p F,1 c =p1-t FC,1 =(300,50,200)-(-15.02,-2,-78.6)=(315.02,52,278.6)mm, pre-approach point p F,1 pre =p F,1 c +50×n1=(315.02+10,52,278.6+49)=(325.02,52,327.6)mm, which provides the target pose for the subsequent robotic arm approach motion.

[0023] The robotic arm motion module sequentially completes pre-approach, normal contact, first stage normal lifting, second stage normal lifting, side-moving opening and return, so that the suction cup smoothly contacts the edge of the toilet seat and the hook pulls the toilet seat ring component to rotate around the hinge to the preset opening angle. The robotic arm motion module includes a pre-approach contact unit, a normal lifting unit, and a side-moving cover opening unit; The pre-approach contact unit executes the robotic arm movement according to the target pose. The robotic arm first moves to the pre-approach point, and then moves linearly at a limited speed along the outer normal of the cover surface to the flange contact position p. F,k c Afterwards, the adsorption is confirmed, and the restricted speed is a preset value lower than the normal movement speed of the robotic arm. This is to prevent the suction cup from impacting the edge of the toilet seat and to ensure that the suction cup makes smooth contact with the toilet seat edge area. The formula for the position of the pre-approach point is: p F,k pre =p F,k c +d pre n k In the formula, p F,k pre This is the pre-approach point before the suction cup contacts the edge of the toilet seat; After adsorption confirmation, the normal lifting unit performs the first stage of normal lifting, raising the toilet seat a first distance along the outer normal direction away from the toilet seat surface, creating an initial gap between the toilet seat and the toilet seat body for the hook to enter. After the hook bears the load and releases pressure, the second stage of normal lifting is performed, continuing to raise the seat a second distance along the same normal direction. The flange position at the end of the two stages of lifting is p. F,k 1 p F,k 2 ;where p F,k 1 =p F,k c +d1n k p F,k 2 =p F,k c +(d1+d2)n k In the formula, p F,k 1 p F,k 2 These are the flange positions at the end of the first and second normal lifting sections, respectively. d1 and d2 are the first and second distances, respectively. d1 is determined based on the thickness of the hook and the gap requirement of the cover edge, and d2 is determined by the safety margin for the suction cup to detach from the toilet seat surface. Preferably, d1 is 20mm and d2 is 10mm. The side-moving lid-opening unit performs side-moving lid opening after two stages of normal lifting, translating along a preset direction in the robot base coordinate system. This causes the chuck to pull the toilet seat ring component to rotate around the hinge. After the lid-opening termination condition is met, the robotic arm returns to the default joint pose. The target flange position at the end of the side-moving lid opening is p. F,k 3 :p F,k 3 =p F,k 2 +d s e s In the formula, p F,k 3 It is the target flange position at the end of the side-sliding cover opening, e s The preset unit direction for the side-opening lid is expressed in B, d. s For along e s lateral displacement, d s >0; preferred e s Take the negative X-axis direction of the base coordinate system, d s The opening angle is set to 250mm. The opening termination condition is that the toilet seat ring component rotates to the preset opening angle. The opening angle is set according to the requirements that the toilet seat ring should not obstruct the operation of the cleaning robot after it is fully opened, and should not collide with the water tank or obstacles behind it.

[0024] The suction cup air path control module manages the switching between the suction channel and the exhaust channel. After the suction cup makes contact, the suction is turned on and the vacuum is confirmed by the negative pressure sensor to determine whether the adsorption is effective. If no effective adsorption is formed, the suction is retracted and the candidate path point is switched. When effective adsorption is formed, the suction is turned off and the exhaust is turned on to actively depressurize, so that the load is quickly transferred from the suction cup to the hook. The suction cup air circuit control module includes an air circuit switching unit and a pressure relief control unit; The air path switching unit manages the switching between the intake and exhaust channels. The intake channel generates negative pressure, allowing the suction cup to adhere to the toilet seat surface. The exhaust channel actively introduces air to break the negative pressure. When the first stage of normal lifting begins, the intake channel is opened and the exhaust channel is closed. The negative pressure sensor is used to perform the first vacuum confirmation. The negative pressure sensor measures the air pressure inside the suction cup in real time. The absolute value of the measured negative pressure is compared with a preset first negative pressure threshold. If the first negative pressure threshold is reached or exceeded, the first vacuum confirmation is passed. Otherwise, the suction cup retracts, and the next candidate waypoint is switched. The first negative pressure threshold is determined based on the minimum negative pressure value required for the maximum load required for the first stage of normal lifting. After the first stage of normal lifting, the negative pressure sensor is read for a second vacuum confirmation. The measured absolute value of negative pressure is compared with the preset second negative pressure threshold. If it reaches or exceeds the second negative pressure threshold, it is determined to be effective adsorption and the second vacuum confirmation is passed. Otherwise, it is determined that no effective adsorption has been formed. The second negative pressure threshold is determined according to the minimum negative pressure value required for the maximum load required for the second stage of normal lifting. When the pressure relief control unit determines that the adsorption is effective, it executes the hook grabbing action. After the hook is in position, it closes the air intake channel and opens the exhaust channel to relieve pressure, so that the load is quickly transferred from the suction cup to the hook. After completion, it closes the air intake and exhaust channels. The hook grabbing action refers to the action of the servo motor moving from the storage position to the grabbing position, so that the hook enters the gap formed by the toilet seat edge and abuts against the edge of the seat. The hook being in position refers to the hook reaching the grabbing position and entering below the edge of the seat. In the pressure relief control unit, the intake channel state is recorded as U. suc U suc A value of 0 indicates that the connection is closed, U. suc A value of 1 indicates the exhaust passage is open; the exhaust passage status is recorded as U. vent U vent A value of 0 indicates that the connection is closed, U. vent A value of 1 indicates the airway is open. Before the adsorption is established and the hook reaches its position, the airflow status is (U). suc U vent )=(1,0); After the load is transferred from the suction cup to the hook, the air path state switches to (U suc U vent = (0,1); After the toilet lid is opened and the pressure is released, the gas path status is (U) = (0,1); suc U vent= (0, 0).

[0025] The hook servo control module drives the hook to switch between the storage position and the hooking position. During the approach, contact and first normal lifting of the suction cup, the hook is kept in storage. After the initial gap is formed, it switches to the hooking position so that the hook enters the gap and takes over the subsequent opening load from the suction cup. During the return phase, the hook is retracted to the storage position. The hook servo control module includes a hook drive unit and an attitude switching unit; The hook drive unit drives the hook to switch between a retracted position and a hooking position. The hook has at least two connecting holes spaced along its longitudinal direction to adjust the length of the working end of the hook relative to the servo shaft. The working end of the hook refers to the bent end of the hook that extends under the toilet seat rim, directly contacts the toilet seat rim, and bears the hooking load. The retracted position is the servo angle position where the hook is retracted and does not obstruct the suction cup from approaching the toilet seat rim. The hooking position is the servo angle position where the hook is extended and the working end enters under the rim of the toilet seat. During the first stage of normal lifting of the suction cup, the pawl remains in the storage position. After the initial gap is formed, the servo motor moves from the storage position to the hooking position, so that the pawl enters the formed gap and abuts against the edge of the cover, taking over the load of subsequent opening of the cover from the suction cup. The value of the servo motor position corresponding to the hooking position depends on the servo motor model and zero position. In the return phase, the servo motor is driven from the hooking position to the storage position. The return phase refers to the final stage where the robot arm returns to the preset joint posture after the lateral opening of the cover reaches the opening termination condition.

[0026] Example 2: Based on the target pose calculated in Example 1, this example completes the entire motion process of suction cup adsorption, hook support, and lateral opening. The preset first-segment normal lifting distance d1 = 20mm, the second-segment normal lifting distance d2 = 10mm, and the lateral opening direction e... s Take the negative X-axis of the base as (-1,0,0), and the lateral displacement distance d s =250mm; First negative pressure threshold -30kPa, second negative pressure threshold -50kPa, limited approach speed 20mm / s; The robotic arm first moves to the pre-approach point (325.02,52,327.6)mm, and then cuts straight in along the outer normal n1 of the cover surface at a limited speed of 20mm / s, reaching the flange contact position (315.02,52,278.6)mm, where the suction cup contacts the toilet seat edge area. During this stage, the servo motor of the hook remains in the retracted position to avoid obstructing the approach of the suction cup; At the start of the first stage of normal lift, the intake passage is opened and the exhaust passage is closed, and the airflow state is (U suc U vent)=(1,0), the negative pressure sensor reads an absolute negative pressure of 35kPa, which reaches and exceeds the first negative pressure threshold of 30kPa, and the first vacuum confirmation is passed; continue to rise 20mm along n1 to the end point p of the first segment. F,1 1 =(315.02,52,278.6)+20×(0.2,0,0.98)=(319.02,52,298.2)mm. At this point, an initial gap is formed between the toilet seat rim and the seat body, allowing the hook to enter. The negative pressure sensor reads an absolute negative pressure value of 55kPa, reaching and exceeding the second negative pressure threshold of 50kPa, thus determining effective adsorption. The hook servo rotates from the storage position to the hooking position, and the working end of the hook extends into the formed gap along the rim and abuts against the rim. After reaching the position, the intake channel is closed and the exhaust channel is opened, and the air path state is switched to (U suc U vent Active pressure relief is performed using (0,1) to quickly transfer the load from the suction cup to the hook. After depressurization is completed, continue to rise 10mm along the same normal direction to the end point of the second stage p. F,1 2 =(315.02,52,278.6)+30×(0.2,0,0.98)=(321.02,52,308)mm, then translate 250mm along the negative X-axis of the base to the lateral displacement endpoint p. F,1 3 =(321.02,52,308)+250×(-1,0,0)=(71.02,52,308)mm, the hook pulls the toilet seat ring component to rotate around the hinge to the preset angle, the opening termination condition is met, the air circuit status is restored to (0,0), the servo motor drives the hook to retract to the storage position, the robotic arm returns to the default joint posture, and this opening operation is completed.

[0027] The state machine decision module uses a hook to hook the permission condition and the side-moving lid opening permission condition for safety interlocking. When the permission condition is met, the system proceeds to the next stage; when it is not met, failure processing is performed, including retracting along the outer normal of the lid and switching to the next candidate waypoint, and returning the toilet lid to its original path. When all K candidate waypoints fail, the system exits the task and issues an alarm. The state machine decision module includes an execution permission unit and a failure handling unit; Upon successful completion of the first and second vacuum verifications, the execution authorization unit allows the grappling hook to enter the hooking position. The hooking authorization condition is C. hook,k =A k Q1Q2; in the formula, A in the formula k Q1 and Q2 are Boolean values ​​that can take the value 0 or 1. When they are multiplied directly, the product is 1 only if all three are 1; if any one of them is 0, the product is 0. kQ1 is the executable flag for the k-th candidate waypoint. It is 1 if the flange target pose corresponding to the current candidate waypoint is within the robot arm's workspace, and 0 otherwise. Q2 is the first vacuum confirmation result; it is 1 if the negative pressure reaches the threshold, and 0 otherwise. Q3 is the second vacuum confirmation result; it is 1 if the negative pressure reaches the threshold, and 0 otherwise. hook,k A value of 0 disallows the hook from entering the hooking position, while a value of 1 allows the hook to enter the hooking position. After the hook enters the hooking position, and after confirming that the air intake channel is closed and the exhaust channel is open, side-sliding cap opening is permitted. The permitted condition for side-sliding cap opening is C. side,k =C hook,k H(1-U suc )U vent In the formula, C hook,k H, (1-U) suc ), U vent The four factors are all either 0 or 1. Multiplying them directly results in a product of 1 only if all four factors are 1; otherwise, the product is 0. (C) side,k To allow the k-th waypoint to perform the side-shift opening, H represents the confirmation result of the grappling hook reaching the hooking position; it is 1 when the servo is in position, and 0 otherwise. C side,k When the value is 0, the lid cannot be opened by sliding it to the side; when the value is 1, the lid can be opened by sliding it to the side. The failure handling unit is in C hook,k If C is 0, the current candidate waypoint is deemed a failure. The intake channel is closed, the exhaust channel is opened, and the system retreats along the outer normal of the cover surface to the pre-approach point. Then, the next candidate waypoint with index k+1 is selected. side,k When the value is 0, the current candidate waypoint is considered to have failed, depressurization and side-moving opening of the toilet seat are prohibited, and the toilet seat is returned to its original path. When all K candidate waypoints have failed, the system exits the task and issues an alarm.

[0028] Example 3: This example describes the safety interlocking and failure handling process of the state machine decision module. Under the current working condition 1, the flange pose with the first path point k=1 is within the workspace after inverse kinematics solution. A k =1, the first vacuum confirmation measurement shows a negative pressure of 35 kPa, exceeding the 30 kPa threshold, Q1=1; the second vacuum confirmation measurement shows a negative pressure of 55 kPa, exceeding the 50 kPa threshold, Q2=1, the hook grabs the permitted condition C. hook,1 =A k ×Q1×Q2=1×1×1=1, allowing the hook to enter the hooking position; after the hook is in position, H=1, and after pressure relief is complete, U suc =0、U vent =1, Side-sliding lid opening permission condition C side,1 =C hook,1 ×H×(1-U suc )×U vent=1×1×1×1=1, allowing side-moving cover opening, the state machine proceeds sequentially until cover opening is complete; Under the current operating condition 2, if the first waypoint k=1 measures a negative pressure absolute value of 20 kPa during the first vacuum confirmation, which does not reach the 30 kPa threshold, and Q1=0, then C hook,1 =1×0×1=0, the current candidate waypoint is determined to be unsuccessful. The intake channel is immediately closed and the exhaust channel is opened to break the negative pressure. The robotic arm retracts along the outer normal of the cover to the pre-approach point. After marking the waypoint as unsuccessful, the next candidate waypoint with index k=2 is automatically selected, and the normal estimation, pose construction and contact adsorption process are re-executed. Under the current operating condition 3, if the hook is not in place during the side-moving cover opening permission stage at the current waypoint (H=0), or if the air path is abnormal after depressurization, resulting in C... side,k =0, side-moving to open the lid is prohibited. The toilet lid is placed back along the original path and the next candidate waypoint is switched. When all 5 candidate waypoints fail in sequence, the state machine determines that the task cannot be completed. The system exits the lid opening task and issues an alarm, prompting the operator to check the toilet lid surface and troubleshoot the problem.

[0029] A toilet lid opening device based on machine vision and suction cup / claw collaboration, the toilet lid opening device includes an electric vacuum suction cup, a servo motor, a servo motor swing arm, a claw, a magnetic metal block, and a mounting structure; the electric vacuum suction cup, servo motor, servo motor swing arm, claw, and magnetic metal block share a common mounting structure as a geometric reference, so that the center of the suction cup, the axis of the servo motor, and the claw maintain a calibrable and fixed relationship; The mounting structure is the support base of the device. The mounting structure has a suction cup mounting surface, a servo motor mounting surface and a metal block mounting surface, which are used to fix the electric vacuum suction cup, the servo motor and the magnetic metal block, respectively.

[0030] The electric vacuum suction cup is fixed to the suction cup mounting surface of the mounting structure, with the suction cup contact surface facing downwards, used to establish a flexible negative pressure connection with the toilet seat edge and complete the initial lifting; the electric vacuum suction cup and the mounting structure are fastened together by screws, so that the position of the suction cup center relative to the mounting structure is fixed. The magnetic metal block is fixed to the metal block mounting surface of the mounting structure and is located at the top of the mounting structure. It is used to cooperate with the electromagnetic holding parts on the robot wrist adapter and the mobile carrier tool position to realize the transfer of the opening end between the robotic arm and the tool parking position. The magnetic metal block is fastened to the mounting structure by screws or embedded fixation. The servo motor is fixed to the servo motor mounting surface of the mounting structure and is located on the side of the mounting structure. The servo motor output shaft faces outward and is used to drive the servo motor swing arm to rotate. The servo motor and the mounting structure are fastened together by at least two screws to fix the position of the servo motor axis relative to the mounting structure. One end of the servo arm is connected to the output shaft of the servo motor and rotates synchronously with the output shaft. The other end is connected to the pawl and is used to transmit the rotational motion of the servo motor to the pawl. The servo arm and the output shaft of the servo motor are fixed together by a shaft hole and a locking screw, so that there is no relative rotation between the servo arm and the output shaft. The hook is connected to the other end of the servo arm. At least two connection holes are provided on the hook along the longitudinal direction of the hook. By selecting different connection holes to connect with the servo arm, the effective length of the working end of the hook relative to the servo shaft can be adjusted. The hook and the servo arm are fastened together by bolts passing through the connection holes. The hook switches between the storage position and the hooking position with the servo arm. The electric vacuum suction cup, servo motor, and magnetic metal block are all directly fixed to the mounting structure. The servo motor swing arm is indirectly mounted to the mounting structure via the servo motor, and the pawl is indirectly mounted to the mounting structure via the servo motor swing arm.

[0031] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A toilet lid opening system based on machine vision and suction cup gripper collaboration, characterized in that: The toilet lid opening system includes a vision perception module, a waypoint and normal generation module, an end pose and tool compensation module, a robotic arm motion module, a suction cup air path control module, a hook servo control module, and a state machine decision module. The visual perception module performs visual detection on the toilet area and outputs an edge mask, a local planar point set, and a candidate waypoint set. The waypoint and normal generation module selects candidate waypoints sequentially from the candidate waypoint set in order, and obtains the out-of-cover normal by fitting the local planar point set of the current candidate waypoint's neighborhood; and generates a pre-approach point along the outer side of the out-of-cover normal. The end-effector pose and tool compensation module determines the end-effector pose based on the outer normal of the cover surface, constructs the three coordinate axes of the flange target coordinate system, and obtains the flange contact position. The robotic arm motion module sequentially completes pre-approach, normal contact, first stage normal lifting, second stage normal lifting, lateral opening of the cover, and return to position; The suction cup air path control module manages the switching between the suction channel and the exhaust channel. It uses a negative pressure sensor to confirm the vacuum and determine whether the adsorption is effective. If no effective adsorption is formed, it retracts and switches the candidate path point. If effective adsorption is formed, it closes the suction and opens the exhaust to actively release pressure. The hook servo control module drives the hook to switch between the storage position and the hooking position. After forming an initial gap, it moves from the storage position to the hooking position so that the hook enters the gap. During the return phase, the hook is retracted to the storage position. The state machine decision module uses a hook to hook the permission condition and the side-moving cover permission condition for safety interlocking. If the permission condition is met, the process proceeds to the next stage; if it is not met, the process fails.

2. The toilet lid opening system based on machine vision and suction cup gripper collaboration according to claim 1, characterized in that: The visual perception module includes a feature extraction unit and a waypoint sorting unit; The feature extraction unit captures an image of the toilet seat and outputs an edge mask, a local planar point set, and a candidate waypoint set. The edge mask is a binary pixel region in the toilet seat image that marks the area where the toilet seat edge is located. The local planar point set is a set of three-dimensional points located on the toilet seat surface within the area near the toilet seat edge. The candidate waypoint set consists of different three-dimensional position points detected at intervals along the toilet seat edge, with each three-dimensional position point serving as a candidate waypoint that can be contacted by the suction cup. The waypoint sorting unit sorts candidate waypoints according to local flatness. Local flatness represents the degree of deviation of toilet seat surface points in the neighborhood of a candidate waypoint relative to the fitted local plane. The toilet seat surface points are three-dimensional points located on the toilet seat surface in the local plane point set. The fitted local plane is a plane representing the local shape of the toilet seat surface obtained by fitting the toilet seat surface points through plane fitting. The degree of deviation is the statistical value of the distance from each toilet seat surface point to the fitted local plane. The candidate waypoints are sorted from high to low local flatness.

3. The toilet lid opening system based on machine vision and suction cup gripper collaboration according to claim 1, characterized in that: The waypoint and normal generation module includes a waypoint selection unit, a normal estimation unit, and a pre-approach point generation unit. The waypoint selection unit selects candidate waypoints sequentially from the candidate waypoints in order of sorting. The index of the currently selected candidate waypoint is denoted as k, and k starts from 1 and increments sequentially, k=1,2,…,K, where K is the total number of valid candidate waypoints. If the current candidate waypoint fails in subsequent operations, it is marked as failed and the next candidate waypoint with index k+1 is selected. The normal estimation unit takes a local planar point set in the neighborhood of the current candidate waypoint and obtains the out-of-cover normal n through planar fitting. k The outer normal of the lid is a unit vector perpendicular to the toilet lid surface and pointing from the toilet lid surface to free space. Free space refers to the open space on the side of the toilet lid surface away from the toilet seat, which is unobstructed and allows the suction cup to approach and lift. The pre-approach point generation unit is located outside the cover surface along the normal direction, at a pre-approach distance d from the candidate waypoint. pre The location is determined as the pre-approach point, where the pre-approach distance d pre It is a preset fixed distance value.

4. The toilet lid opening system based on machine vision and suction cup gripper collaboration according to claim 1, characterized in that: The end-effector pose and tool compensation module includes a pose determination unit, a coordinate system construction unit, and a tool compensation unit. The attitude determination unit determines the attitude of the end-effector based on the out-of-surface normal of the toilet seat, and the tool axis z T,k The positive direction is fixed as from the flange origin to the suction cup contact surface, and the positive direction of the outer normal of the cover surface is fixed as from the cover surface to free space, where the tool axis z T,k Let B be the unit direction vector of the tool axis in the flange target coordinate system of the k-th candidate waypoint; The coordinate system construction unit selects a reference vector that is not parallel to the tool axis, and constructs a right-handed orthogonal coordinate system through the cross product of vectors to obtain x. T,k y T,k , and z T,k The three coordinate axes that together constitute the target coordinate system of the flange, where x T,k Let y be the unit direction vector of the X-axis in the target coordinate system of the flange. T,k The unit direction vector of the Y-axis in the target flange coordinate system; The tool compensation unit calculates the tool offset vector t from the origin of the flange target coordinate system to the suction cup contact center point. FC,k Then, the flange contact position p corresponding to the kth candidate waypoint is calculated. F,k c .

5. A toilet lid opening system based on machine vision and suction cup gripper collaboration as described in claim 1, characterized in that: The robotic arm motion module includes a pre-approach contact unit, a normal lifting unit, and a side-moving cover opening unit; The pre-approach contact unit executes the robotic arm movement according to the target pose. The robotic arm first moves to the pre-approach point, and then moves along the outer normal of the cover surface to the flange contact position p. F,k c Then, adsorption was confirmed. After adsorption confirmation, the normal lifting unit performs the first stage of normal lifting, lifting a first distance along the outer normal of the lid towards the side away from the toilet lid, so that an initial gap is formed between the toilet lid and the toilet seat body, allowing the hook to enter; after the hook bears the load and releases the pressure, the second stage of normal lifting is performed, continuing to lift a second distance along the same normal. The side-moving lid-opening unit performs side-moving lid opening after two stages of normal lifting, translating along a preset direction in the robot base coordinate system. This causes the chuck to pull the toilet seat ring component to rotate around the hinge. After the lid-opening termination condition is met, the robotic arm returns to the default joint pose. The target flange position at the end of the side-moving lid opening is p. F,k 3 The opening termination condition is that the toilet seat ring component rotates to a preset opening angle.

6. A toilet lid opening system based on machine vision and suction cup gripper collaboration as described in claim 1, characterized in that: The suction cup air circuit control module includes an air circuit switching unit and a pressure relief control unit; The air path switching unit manages the switching between the intake channel and the exhaust channel. The intake channel is an air path that generates negative pressure and causes the suction cup to adhere to the toilet seat surface, while the exhaust channel is an air path that actively introduces air to break the negative pressure. When the first normal lift begins, the intake channel is opened and the exhaust channel is closed. The negative pressure sensor is read to perform the first vacuum confirmation. The measured absolute value of the negative pressure is compared with the preset first negative pressure threshold. When the first negative pressure threshold is reached or exceeded, the first vacuum confirmation is passed. Otherwise, the suction cup retracts and then the next candidate waypoint is switched. After the first normal lift, the negative pressure sensor is read for a second vacuum confirmation. The measured absolute value of the negative pressure is compared with the preset second negative pressure threshold. When the second negative pressure threshold is reached or exceeded, it is determined to be effective adsorption and the second vacuum confirmation is passed. Otherwise, it is determined that no effective adsorption has been formed. When the pressure relief control unit determines that the adsorption is effective, it executes the hook to pick up the product. After the hook is in place, it closes the air intake channel and opens the air exhaust channel to relieve pressure. After completion, it closes the air intake and air exhaust channels. The hook picking refers to the action of the servo moving from the storage position to the hook position, so that the hook enters the gap formed by the toilet seat edge and abuts against the edge of the seat. The hook being in place refers to the hook reaching the hook position and entering below the edge of the seat.

7. A toilet lid opening system based on machine vision and suction cup gripper collaboration as described in claim 1, characterized in that: The hook servo control module includes a hook drive unit and an attitude switching unit; The hook drive unit drives the hook to switch between a retracted position and a hooking position. The hook has at least two connection holes spaced at intervals along its longitudinal direction. The working end of the hook refers to the bent end of the hook that is used to extend under the edge of the toilet seat, directly contact the edge of the toilet seat, and bear the hooking load. The retracted position is the servo angle position where the hook is retracted and does not obstruct the suction cup from approaching the edge of the toilet seat. The hooking position is the servo angle position where the hook is extended and the working end enters under the edge of the seat. During the first stage of normal lifting of the suction cup, the pawl remains in the storage position. After the initial gap is formed, the servo motor moves from the storage position to the hooking position, so that the pawl enters the formed gap and abuts against the edge of the cover, taking over the load of subsequent opening of the cover from the suction cup. The value of the servo motor position corresponding to the hooking position depends on the servo motor model and zero position. In the return phase, the servo motor is driven from the hooking position to the storage position. The return phase refers to the final stage where the robot arm returns to the preset joint posture after the lateral opening of the cover reaches the opening termination condition.

8. A toilet lid opening system based on machine vision and suction cup gripper collaboration as described in claim 1, characterized in that: The state machine decision module includes an execution permission unit and a failure handling unit; Upon successful completion of the first and second vacuum verifications, the execution authorization unit allows the grappling hook to enter the hooking position. The hooking authorization condition is C. hook,k =A k Q1Q2; In the formula, A k The three factors, Q1, Q2, and Q3, are all Boolean values ​​that can take the value 0 or 1. Multiplying them directly results in a product of 1 when all three factors are 1, and a product of 0 when one factor is 0. k This is the executable flag for the k-th candidate waypoint, Q1 is the result of the first vacuum confirmation, Q2 is the result of the second vacuum confirmation, and C... hook,k A value of 0 disallows the hook from entering the hooking position, while a value of 1 allows the hook to enter the hooking position. After the hook enters the hooking position, and the air intake channel is closed and the exhaust channel is opened, side-sliding opening of the cover is permitted under condition C. side,k =C hook,k H(1-U suc )U vent In the formula, C hook,k H, (1-U) suc ), U vent The four factors are all either 0 or 1. Multiplying them directly results in a product of 1 when all four factors are 1, and a product of 0 when one factor is 0. C side,k To allow the k-th waypoint to perform the side-movement opening, H is the confirmation result of the hook reaching the hooking position, and C... side,k When the value is 0, the lid cannot be opened by sliding it to the side; when the value is 1, the lid can be opened by sliding it to the side. The failure handling unit is in C hook,k When C is 0, the current candidate waypoint is deemed a failure. The intake channel is closed, the exhaust channel is opened, and the system retreats along the outer normal of the cover surface to the pre-approach point. Then, the next candidate waypoint with index k+1 is selected. side,k When the value is 0, the current candidate waypoint is considered to have failed, and the toilet seat is returned to its original path. When all K candidate waypoints have failed, the system exits the task and issues an alarm.

9. A toilet lid opening device based on machine vision and suction cup claw collaboration, characterized in that: The toilet lid opening device includes an electric vacuum suction cup, a servo motor, a servo motor swing arm, a claw, a magnetic metal block, and a mounting structure; the electric vacuum suction cup, servo motor, servo motor swing arm, claw, and magnetic metal block share the mounting structure as a geometric reference; The mounting structure is the support base of the device. The mounting structure has a suction cup mounting surface, a servo motor mounting surface and a metal block mounting surface, which are used to fix the electric vacuum suction cup, the servo motor and the magnetic metal block, respectively. The electric vacuum suction cup is fixed to the suction cup mounting surface of the mounting structure, with the suction cup contact surface facing downwards. The electric vacuum suction cup and the mounting structure are fastened together by screws, and the position of the center of the suction cup relative to the mounting structure is fixed. The magnetic metal block is fixed to the metal block mounting surface of the mounting structure and is located at the top and middle of the mounting structure. The magnetic metal block and the mounting structure are fastened together by screws. The servo motor is fixed to the servo motor mounting surface of the mounting structure and is located on the side of the mounting structure. The servo motor output shaft faces outward, and the servo motor is connected to the mounting structure by screws. One end of the servo arm is connected to the output shaft of the servo motor, and the other end is connected to the pawl. The servo arm and the output shaft of the servo motor are fixed together by screws through shaft holes. The hook is connected to the other end of the servo arm. At least two connection holes are provided on the hook along the longitudinal direction of the hook. The hook is connected to the servo arm through the corresponding connection holes. The hook and the servo arm are fastened together by bolts passing through the connection holes. The hook switches between the storage position and the hooking position with the servo arm. The electric vacuum suction cup, servo motor, and magnetic metal block are all directly fixed to the mounting structure. The servo motor swing arm is indirectly mounted to the mounting structure via the servo motor, and the pawl is indirectly mounted to the mounting structure via the servo motor swing arm.

Images