Method and system for body intelligent wheeled mobile humanoid robot to remove packing tape

Abstract

The application provides a method and system for a body intelligent wheeled mobile humanoid robot to remove a packing belt, applied to the technical field of robot control, determines the pose of the packing belt and the operation parameters of the double mechanical arms according to the visual sensing information of the object to be removed, controls the first mechanical arm to clamp and lift the packing belt, adjusts the lifting state according to the force information in the lifting process, separates the packing belt from the object to be removed, controls the second mechanical arm to cut the packing belt when the first mechanical arm keeps the lifting state, and determines the cutting state of the packing belt in combination with the force information change in the cutting process and the visual detection result, thereby solving the problems of low success rate, poor safety and insufficient universality of the flexible packing belt removal in the prior art.

Description

Technical Field

[0001] This application relates to the field of robot control technology, specifically to a method and system for removing packing straps using an embodied intelligent wheeled mobile humanoid robot. Background Technology

[0002] In industrial production and logistics transportation, packing straps are commonly used to secure packages and other objects, improving their stability during handling, storage, and transportation. Before unpacking or subsequent processing, the packing straps need to be cut and removed from the package. Currently, the following methods are mainly used for removing packing straps: 1. Manual dismantling, which usually involves workers cutting the packing straps with scissors. This method is labor-intensive, inefficient, poses a safety hazard of cuts from the scissors, and results in unstable work positions due to high staff turnover. 2. Single-arm robotic dismantling typically involves mounting a cutting gripper and a 2D camera at the end of the robotic arm. The position of the packing strap is first photographed and located, and then the gripper is moved to cut. However, since the packing strap is a flexible material, it is prone to slipping during cutting, leading to cutting failure. The success rate is only about 70%. A fixed teaching position and fixed motion path are usually used. When the placement of the object to be dismantled or the position of the packing strap changes, it is necessary to re-teach, which has poor adaptability. 3. General dual-arm robot dismantling typically performs clamping and cutting operations according to a preset sequence of actions. It can only handle packing straps with uniform tension and fixed position. When the position or tension of the packing strap changes, the box may shake or the packing strap may spring back and injure people due to the release of tension during the cutting process. Since the packing strap is a flexible object, the sliding and tension changes that occur during the cutting process will also affect the stability of the collaborative operation of the two robotic arms. Furthermore, the completion of the cutting action by the cutting tool does not necessarily mean that the packing strap has been completely cut. It is difficult to accurately determine the actual cutting state of the packing strap based solely on the action of the cutting tool, which may lead to the robot entering the subsequent operation process when the packing strap has not been cut or has not been completely cut.

[0003] Therefore, a new mobile dual-arm robot solution for the collaborative removal of flexible packing straps is needed. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a method and system for removing packing straps using an embodied intelligent wheeled humanoid robot. By coordinating the timing of the first robotic arm picking up the strap and the second robotic arm cutting, combined with force feedback adjustment during the picking process and force and visual detection during the cutting process, the stability of the flexible packing strap picking and cutting process and the reliability of the cutting state judgment are significantly improved.

[0005] The embodiments in this specification provide the following technical solutions: This specification provides an embodiment of a method for removing packing straps from a humanoid intelligent wheeled mobile robot, including: Obtain visual perception information of the object to be demolished; The packing strap pose information is determined based on the visual perception information; Based on the packing strap pose information, determine the packing strap picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm; The first robotic arm is controlled to lift the packing strap according to the picking operation parameters, so that the packing strap is separated from the object to be removed; Acquire the first force information of the first robotic arm during the lifting process, and adjust the state of the packing strap according to the first force information; While the first robotic arm remains in the lifted state, the second robotic arm is controlled to cut the packing strap according to the cutting operation parameters; The second force information of the first robotic arm during the cutting process and the visual detection results of the packing strap during the cutting process are obtained to determine the cutting state of the packing strap.

[0006] This specification also provides an embodiment of a system for removing packing straps from an intelligent wheeled humanoid robot, comprising: a mobile composite robot body; The mobile composite robot body includes a mobile chassis, a first robotic arm, a second robotic arm, and a controller, wherein the first robotic arm and the second robotic arm are mounted on the mobile chassis; A vision sensor, located at the end of the first robotic arm, is used to acquire visual perception information of the object to be dismantled and visual detection results of the packing strap during the cutting process. A clamping assembly, disposed at the end of the first robotic arm, is used to clamp and lift the packing strap; A force sensor is installed at the end of the first robotic arm to acquire information on the first force applied by the first robotic arm during the picking process and information on the second force applied during the shearing process. A cutting component, located at the end of the second robotic arm, is used to cut the packing strap; The controller is connected to the first robotic arm, the second robotic arm, the vision sensor, the force sensor, the clamping assembly, and the shearing assembly, respectively. The controller is used to perform the following operations: The packing strap pose information is determined based on the visual perception information; Based on the packing strap pose information, determine the packing strap picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm; The first robotic arm is controlled to clamp and lift the packing strap according to the strapping operation parameters, thereby separating the packing strap from the object to be removed. The first robotic arm's operation on the packing strap is adjusted based on the first force information; While the first robotic arm holds the packing strap up, the second robotic arm is controlled to cut the packing strap using the cutting assembly according to the cutting operation parameters. Based on the change in the second force information and the visual detection result, the cutting state of the packing strap is determined.

[0007] Compared with the prior art, the technical effects that can be achieved by at least one of the above-mentioned technical solutions adopted in the embodiments of this specification include at least the following: 1. The first robotic arm first clamps, lifts, and separates the packing strap from the object to be removed. The second robotic arm performs cutting while the first robotic arm is in the picking state. This significantly reduces the impact of the packing strap being close to the object to be removed or its position shifting during the cutting process on the cutting operation, and improves the stability of the picking and cutting process. 2. Adjust the strap-picking state of the first robotic arm according to the first force information during the strap-picking process, so that the first robotic arm can adjust the strap-picking operation according to the force of the strap, thereby improving the adaptability to changes in the force of the strap. 3. The cutting state is determined based on the changes in the second force during the cutting process and the visual detection results. The reliability of the cutting state judgment is improved by judging whether the packing tape has been cut from both force and visual dimensions. Attached Figure Description

[0008] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating a method for removing packing straps using a humanoid intelligent wheeled mobile robot according to a specific embodiment of this application. Figure 2 This is a hierarchical control architecture diagram of a system for removing packing straps using an intelligent wheeled humanoid robot according to a specific embodiment of this application; Figure 3 This is a schematic diagram of a robot working next to a container in a specific embodiment of this application; Figure 4 This is the mobile composite robot body in a specific embodiment of this application; Figure 5 This is a schematic diagram of the structure of the first robotic arm gripper assembly in a specific embodiment of this application; Figure 6 This is a schematic diagram of the second robotic arm gripper assembly structure in a specific embodiment of this application; Figure 7 This is a schematic diagram of the mobile chassis component structure in a specific embodiment of this application. Detailed Implementation

[0010] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0011] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. This application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, in the absence of conflict, the following embodiments and features in the embodiments can be combined with each other. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0012] It should be noted that various aspects of embodiments within the scope of the appended claims are described below. It will be apparent that the aspects described herein can be embodied in a wide variety of forms, and any particular structure and / or function described herein is merely illustrative. Based on this application, those skilled in the art will understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects set forth herein can be used to implement the device and / or practice the method. Additionally, this device and / or method can be implemented using structures and / or functionalities other than one or more of the aspects set forth herein.

[0013] It should also be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. The drawings only show the components related to this application and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the shape, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0014] Additionally, specific details are provided in the following description to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that practice can be carried out without these specific details.

[0015] Regarding existing flexible strapping robot removal solutions, such as the prior art mentioned in the background section, in the process of analyzing related problems and exploring solutions, it was found that: the position and stress state of the flexible strapping are prone to change during operation, and existing solutions are difficult to maintain a stable state suitable for shearing; at the same time, the execution state of the shearing action is not completely consistent with the actual shearing state of the strapping, making it difficult to reliably determine whether the removal is complete.

[0016] Based on this, this specification proposes a mobile dual-arm robot collaborative removal scheme for flexible packing straps. The overall idea is as follows: the packing strap pose and the working parameters of the two robotic arms are determined based on the visual perception information of the object to be removed. The first robotic arm is controlled to grip and lift the packing strap. The lifting state is adjusted according to the force information during the lifting process to separate the packing strap from the object to be removed. While the first robotic arm is in the lifting state, the second robotic arm is controlled to cut the packing strap. The cutting state of the packing strap is determined by combining the force information changes during the cutting process and the visual detection results. This solves the problems of low success rate, poor safety, and insufficient versatility in the removal of flexible packing straps in existing solutions.

[0017] The technical solutions provided by the various embodiments of this application are described below with reference to the accompanying drawings.

[0018] like Figure 1 As shown in the embodiments of this specification, a method for removing packing straps from a humanoid intelligent wheeled mobile robot is provided, including: Step S10: Obtain visual perception information of the object to be dismantled, and determine the packing strap pose information based on the visual perception information.

[0019] Specifically, such as Figure 2 As shown, visual perception information of the object to be dismantled is obtained through the environmental perception component. The visual perception information can be collected by a three-dimensional (3D) vision sensor installed at the end of the first robotic arm. The collected visual perception information includes image information of the object to be dismantled and its surface packing straps. The visual perception information is processed to identify the position and posture of the packing straps relative to the object to be dismantled, thereby obtaining the packing strap pose information. This packing strap pose information is then provided to the subsequent operation parameter determination process.

[0020] Step S20: Based on the packing strap pose information, determine the packing operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm.

[0021] Specifically, during the task decision-making and motion planning process, based on the position and posture of the packing strap relative to the object to be removed, the picking operation parameters required for the first robotic arm to perform the packing strap approach, clamping and picking operations, and the cutting operation parameters required for the second robotic arm to approach the packing strap and perform the cutting operation are determined respectively.

[0022] The picking parameters are used to control the first robotic arm to reach the picking position corresponding to the packing strap, and the cutting parameters are used to control the second robotic arm to reach the cutting position corresponding to the packing strap, thereby establishing the division of labor and action sequence between the first and second robotic arms.

[0023] Step S30: Control the first robotic arm to clamp and lift the packing strap according to the picking operation parameters, so that the packing strap is separated from the object to be removed.

[0024] Specifically, the first robotic arm is controlled to move the clamping component close to the packing strap according to the picking operation parameters, so that the clamping component enters the position where it can clamp the packing strap. Then, the clamping component is controlled to close to clamp the packing strap. After clamping is completed, the first robotic arm is controlled to move the packing strap away from the surface of the object to be removed, so that the packing strap is picked up and separated from the object to be removed, thereby forming a working space between the packing strap and the object to be removed for the second robotic arm to perform cutting.

[0025] Step S40: Obtain the first force information of the first robotic arm during the strap picking process, and adjust the strap picking state of the first robotic arm on the packing strap according to the first force information.

[0026] Specifically, a force sensor installed at the end of the first robotic arm detects the force exerted on the first robotic arm during the clamping and lifting of the packing strap, and obtains the first force information. Based on the force state between the first robotic arm and the packing strap as represented by the first force information, it is determined whether the current strap lifting state is suitable for subsequent cutting. When the strap lifting state deviates, the motion state of the first robotic arm is adjusted to keep the packing strap separated from the object to be removed, thus providing stable operating conditions for the second robotic arm to perform cutting.

[0027] Step S50: While the first robotic arm is in the picking state, the second robotic arm is controlled to cut the packing strap according to the cutting operation parameters.

[0028] Specifically, after determining that the first robotic arm has gripped and lifted the packing strap, the first robotic arm is controlled to continue to maintain the current strap lifting state, while the second robotic arm is controlled to drive the cutting component to approach the packing strap and perform cutting according to the cutting operation parameters.

[0029] The cutting component can be a pneumatic scissor set at the end of the second robotic arm. By having the first robotic arm and the second robotic arm work together in the order of picking up the strap first and cutting it later, the cutting can be completed while the strapping is separated from the object to be removed and is constrained by the first robotic arm.

[0030] Step S60: Obtain the second force information of the first robotic arm during the cutting process, and the visual detection result of the packing strap during the cutting process. Based on the change of the second force information and the visual detection result, determine the cutting state of the packing strap.

[0031] Specifically, during the process of the second robotic arm cutting the packing strap, the force sensor at the end of the first robotic arm continuously detects the force acting on the first robotic arm to obtain the second force information; at the same time, the visual detection results of the packing strap during the cutting process are obtained through a vision sensor. Based on the change of the second force information during the cutting process, and combined with the continuous state of the packing strap represented by the visual detection results, it is determined whether the packing strap is in a cut state or an uncut state, thus obtaining the cutting state judgment result of this collaborative dismantling operation.

[0032] This embodiment reduces the impact of flexible strapping slippage or positional changes on the cutting operation by forming a shearing space that separates the strapping from the surface of the object to be removed; it improves the stability of the coordinated operation of strapping and cutting by adaptively adjusting the strapping state according to the force changes during the strapping process; and it significantly improves the reliability of the shearing state judgment by judging the cutting state from both force and visual dimensions, reducing misjudgments caused by relying solely on the shearing action.

[0033] In some embodiments, determining the picking parameters of the first robotic arm and the cutting parameters of the second robotic arm based on the packing strap pose information includes: The physical attribute information of the packing tape is determined based on the visual perception information, wherein the physical attribute information includes at least one of color and bandwidth; Based on the packing strap pose information and the physical attribute information, the picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm are determined.

[0034] Specifically, such as Figure 2 As shown, the environmental perception and feature extraction layer processes visual perception information and extracts at least one physical attribute from the color and bandwidth of the packing tape.

[0035] The color of the packing tape can be identified by visual color segmentation, such as transparent, yellow, or blue; the width of the packing tape can be determined by edge width measurement, such as 12mm, 15mm, or 19mm.

[0036] It should be noted that the colors and bandwidths mentioned above are just examples. In actual processing, the corresponding recognition results are determined based on the physical properties of the packing tape reflected by the visual perception information.

[0037] The pose and physical properties of the packing strap are input into the adaptive task decision layer. Based on the pose and physical properties of the packing strap, the picking parameters of the first robotic arm and the cutting parameters of the second robotic arm are determined respectively.

[0038] For example, the gripper's picking force is adjusted according to the color and width of the packing strap to obtain the picking operation parameters of the first robotic arm; the scissor opening size and cutting speed are adjusted according to the color and width of the packing strap to obtain the cutting operation parameters of the second robotic arm.

[0039] The first robotic arm grips and lifts the packing strap according to the picking operation parameters, while the second robotic arm cuts the packing strap according to the cutting operation parameters.

[0040] This embodiment identifies the color and width of the packing strap and adjusts the gripper's picking force, the scissor's opening size, and the cutting speed accordingly. This adapts the operating parameters of the dual robotic arms to the actual physical properties of the packing strap, reducing instability in picking and cutting failures caused by excessively slippery or thick packing straps, and improving the robot's adaptability to different types of packing straps.

[0041] In some embodiments, the method further includes: Based on the visual perception information, determine the spatial topology information of the packing tape; Based on the spatial topology information, determine the bundling pattern of the packing strap; When multiple packing straps exist on the object to be dismantled, the dismantling priority among the multiple packing straps is determined according to the binding pattern; The multiple packing straps were removed according to the removal priority.

[0042] Specifically, skeleton extraction and corner detection are performed on the visual perception information to obtain spatial topological information representing the extension direction, arrangement, and intersection relationships of the packing straps. Based on the spatial topological information, the spatial distribution state of the packing straps is identified to determine the binding pattern. The binding patterns of the packing straps can include cross-shaped, parallel, and vertical wrapping patterns, etc.

[0043] When there are multiple packing straps on the object to be dismantled, the dismantling priority among the multiple packing straps is determined according to the binding pattern of each packing strap and the spatial topological relationship between the multiple packing straps.

[0044] For example, for cross-shaped packing straps, the removal priority of the longitudinal packing straps can be set higher than that of the transverse packing straps. Remove the longitudinal packing straps first, and then remove the transverse packing straps to reduce the chance of the package coming apart during the removal process.

[0045] For example, for parallel packing straps, the removal priority of each packing strap can be set from the outside to the inside.

[0046] After determining the removal priority among multiple packing straps, the packing straps to be removed are selected sequentially according to the removal priority. The packing strap position information and the working parameters of the two robotic arms are determined for the currently selected packing strap. The first robotic arm performs the clamping and picking operation, and the second robotic arm performs the cutting operation. After the removal of the current packing strap is completed, the next packing strap is selected according to the removal priority to continue the removal until the removal of multiple packing straps is completed.

[0047] This embodiment identifies the spatial topology information and bundling pattern of the packing straps to determine the removal priority among multiple packing straps, so that multiple packing straps are removed in sequence according to the order that matches the bundling relationship. This reduces the situation where the object to be removed is scattered or subsequent packing straps are difficult to handle due to an unreasonable removal order, thereby improving the stability of the continuous removal process of multiple packing straps.

[0048] In some embodiments, adjusting the operation of the first robotic arm on the packing strap based on the first force information includes: The tightness of the packing strap is determined based on the first force information; The clamping force of the first robotic arm on the packing strap is adjusted according to the tightness to maintain the first robotic arm's picking state of the packing strap.

[0049] Specifically, such as Figures 3 to 5 As shown, during the process of the first robotic arm clamping and lifting the packing strap, the force sensor 3 set at the end of the first robotic arm detects the force between the first robotic arm and the packing strap to obtain the first force information; the tightness of the packing strap is determined according to the first force information, and the clamping force of the clamping component at the end of the first robotic arm is adjusted accordingly to make the clamping force match the current tightness of the packing strap.

[0050] After the clamping force is adjusted, the first robotic arm continues to clamp and lift the packing strap, keeping the packing strap in a lifted state separated from the object to be removed, thus providing a stable packing strap state for the second robotic arm to perform cutting.

[0051] It should be noted that the appropriate force detection method can be selected based on the hardware conditions of the mobile dual-arm robot. For example, a six-dimensional force sensor can be installed at the end of the first robotic arm to detect the force exerted on the end of the first robotic arm during the gripping and lifting of the packing strap, thus obtaining the initial force information. Alternatively, a joint torque sensor can be used to detect the force exerted by the first robotic arm during the strap-lifting process, and the initial force information can be obtained based on the detection results of the joint torque sensor. Selecting the appropriate force detection method based on the hardware conditions can balance the accuracy of force detection and the cost of hardware configuration, providing force information for adjusting the strap-lifting state under different hardware configurations.

[0052] This embodiment adapts the clamping force to the actual tension of the strapping, reducing instability in the strapping state caused by clamping force mismatch, and improving the adaptability of the first robotic arm to strapping with different tension states and the stability of the strapping process.

[0053] In some embodiments, the method further includes: The force changes during the lifting process are determined based on the first force information; When the force change represents a sudden change in force, the visual detection results of the packing strap during the strap picking process are obtained; When it is determined that the packing strap has slipped based on the visual detection results during the packing strap picking process, the packing strap pose information is re-determined based on the visual detection results during the packing strap picking process. Based on the redefined strapping pose information, the strapping operation parameters are updated, and the first robotic arm is controlled to clamp and lift the strapping again according to the updated strapping operation parameters.

[0054] Specifically, during the process of the first robotic arm gripping and lifting the packing strap, the force sensor detects the force acting on the first robotic arm in real time to obtain the first force information; the force changes during the lifting process are determined based on the continuously obtained first force information. When the force change indicates a sudden change in force, the visual detection result corresponding to the gripping position of the packing strap and the first robotic arm is obtained.

[0055] Based on the visual detection results, it is determined whether the strapping has detached from the gripping position of the first robotic arm. When the visual detection results indicate that the strapping has slipped, the position and posture of the strapping are re-identified based on the visual detection results to obtain new strapping pose information. Then, the strapping picking operation parameters of the first robotic arm are updated based on the re-determined strapping pose information so that the updated strapping operation parameters correspond to the actual pose of the strapping after slippage.

[0056] After updating the strapping operation parameters, the first robotic arm is controlled to move back to the corresponding strapping position according to the updated strapping operation parameters, and clamps and lifts the strapping again, so that the strapping is separated from the object to be removed, thus restoring the first robotic arm's strapping state.

[0057] This embodiment identifies abnormal states during the strapping process by changing the force applied, and uses visual detection results to confirm whether the strapping has slipped, reducing misjudgments caused by judging slippage solely based on sudden changes in force. After confirming that the strapping has slipped, the strapping posture information is redefined and the strapping operation parameters are updated, so that the first robotic arm performs clamping and lifting again according to the actual posture of the strapping, forming a closed loop for recovery processing after strapping slippage, which significantly improves the stability of the strapping process.

[0058] In some embodiments, determining the cut state of the packing strap includes: Based on the second force information, determine the change in force acting on the first robotic arm; When the force change characterizes the sudden drop in the force on the first robotic arm, an initial cutting determination result characterizing that the packing strap has been cut is obtained; Based on the visual inspection results, the initial cutting determination results are verified to determine the cutting state of the packing strap.

[0059] Specifically, during the process of the first robotic arm maintaining the strapping state and the second robotic arm cutting the packing strap, the force sensor set at the end of the first robotic arm detects the force acting on the first robotic arm in real time to obtain the second force information. Based on the continuously obtained second force information, the force change of the first robotic arm during the cutting process is determined.

[0060] When the strapping is cut, its original tension changes, and the tension on the first robotic arm decreases accordingly. An initial cut determination result is obtained when the force on the first robotic arm suddenly drops, indicating that the strapping has been cut. This initial cut determination result indicates that the strapping may have been cut based on the force change, and is subsequently verified by visual inspection.

[0061] After obtaining the initial cut determination result, the visual inspection results of the strapping during the cutting process are used to determine whether the strapping remains continuous. If the visual inspection result indicates that the strapping has broken, the initial cut determination result is verified to be consistent with the actual state of the strapping, and the strapping is determined to be in a cut state. If the visual inspection result indicates that the strapping remains continuous, the initial cut determination result is determined to have failed the verification, and the strapping is determined to be in an uncut state.

[0062] This embodiment uses the change in the second force information to form an initial shearing judgment result, and then uses the visual detection result to verify the initial shearing judgment result, reducing misjudgments caused by judging the shearing state solely based on the change in force or solely based on the visual detection result, and improving the reliability of the shearing state judgment of the packing tape.

[0063] In some embodiments, the method further includes: Based on the visual perception information, determine the size information of the object to be demolished; Perform at least one of the following operations based on the size information: Determine the end-effector approach postures of the first and second robotic arms; Adjust the workspace boundaries of the first robotic arm and the second robotic arm.

[0064] Specifically, such as Figure 3 and Figure 5 As shown, visual perception information of the object to be demolished is obtained through visual sensor 1 (such as a 3D camera). The length, width and height of the object to be demolished are determined based on the visual perception information to obtain the size information of the object to be demolished. The object to be demolished is classified into size levels according to its length, width and height, such as small, medium or large objects. The determined size information or size level is used for the operation control of the dual robotic arms.

[0065] In one embodiment, the end-effector approach postures of the first robotic arm and the second robotic arm are determined based on the size information of the object to be dismantled. The end-effector approach posture of the first robotic arm is used to bring the gripping component closer to the packing strap from a direction adapted to the size of the object to be dismantled. The end-effector approach posture of the second robotic arm is used to bring the cutting component closer to the cutting position from a direction adapted to the size of the object to be dismantled. The first robotic arm and the second robotic arm perform the picking operation and the cutting operation respectively according to their respective end-effector approach postures.

[0066] In another implementation, the workspace boundaries of the first and second robotic arms are adjusted according to the size information of the object to be dismantled, so that the range of motion of the two robotic arms is adapted to the space occupied by the object to be dismantled. The workspace boundaries are used to limit the range of motion of the first and second robotic arms when performing the picking and cutting operations, reducing the possibility of collision between the robotic arm ends and the object to be dismantled. The determination of the end-approach posture and the adjustment of the workspace boundaries can be performed individually or simultaneously.

[0067] This embodiment determines the end-effector approach posture of the dual robotic arms or adjusts the workspace boundary of the dual robotic arms based on the size information of the object to be dismantled, so that the working space of the dual robotic arms is adapted to the actual size of the object to be dismantled. This reduces the problem that fixed approach postures or fixed workspaces are difficult to adapt to objects of different sizes, reduces the risk of collision between the end-effector of the robotic arms and the object to be dismantled, and reduces the need for manual teaching of the robotic arms when switching between objects of different sizes.

[0068] In some implementations, acquiring visual perception information of the object to be dismantled, and determining the packing strap pose information based on the visual perception information, includes: A global scan of the object to be demolished is performed to obtain global visual perception information; Based on the global visual perception information, determine the pose information of the object to be demolished; Based on the pose information of the object to be dismantled and the preset pose mapping relationship, the initial position information of the packing strap is determined. The pose mapping relationship is used to characterize the correspondence between the pose of the object to be dismantled and the initial position of the packing strap. The local scanning area is determined based on the initial position information, and the local scanning area is scanned to obtain the local visual perception information of the packing tape. The packing strap pose information is determined based on the local visual perception information.

[0069] Specifically, a vision sensor installed at the end of the first robotic arm performs a global scan of the object to be demolished, collects visual information about the object and its surface, obtains global visual perception information, identifies the position and posture of the object to be demolished in the work space based on the global visual perception information, and obtains the pose information of the object to be demolished.

[0070] A pre-set box pose-packing strap position mapping table is used to represent the correspondence between the pose of the object to be dismantled and the initial position of the packing strap. Based on the pose information of the object to be dismantled, the box pose-packing strap position mapping table is queried to determine the initial position of the packing strap relative to the object, thus obtaining the initial position information of the packing strap.

[0071] Based on the initial position information of the packing strap, a local scanning area is determined. The first robotic arm is controlled to move a vision sensor, such as a color 3D camera, above the packing strap to perform a high-definition scan of the packing strap within the local scanning area, thereby obtaining local visual perception information of the packing strap. Based on the local visual perception information, the position and posture of the packing strap within the local scanning area are identified, and packing strap pose information with higher accuracy than the initial position information is obtained.

[0072] The strapping pose information is used as the basis for determining the strapping picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm. The first robotic arm corrects the initial strapping picking path based on the strapping pose information, so that the gripping component approaches the strapping along the corrected picking path; the second robotic arm determines the cutting position corresponding to the strapping based on the strapping pose information.

[0073] This embodiment determines the pose of the object to be dismantled through global scanning, obtains the initial position of the packing strap using the box pose-packing strap position mapping table, and then performs local high-definition scanning based on the initial position, forming a visual positioning process that combines global preliminary positioning with local fine positioning. This reduces the area of ​​local high-definition scanning, improves the accuracy of determining the packing strap pose information, and provides position and attitude basis for determining the working parameters of the dual robotic arms and correcting the initial path.

[0074] In some embodiments, the method further includes: After determining that the packing strap is cut, the second robotic arm is controlled to exit the cutting position; The first robotic arm is controlled to maintain its grip on the packing strap and to pull the cut packing strap to a preset recycling position along a preset trajectory. Control the first robotic arm to release the packing strap.

[0075] Specifically, after determining that the packing strap is in a cut state based on the change in the second force information and the visual detection results, the second robotic arm is controlled to immediately drive the shearing component out of the shearing position and retract the shearing component to a safe position, so that the shearing component is far away from the cut packing strap and the collision between the packing strap and the shearing component when it springs back is reduced.

[0076] After the second robotic arm retracts from the cutting position, the first robotic arm continues to hold the packing strap to prevent the cut straping strap from falling off. The first robotic arm is controlled to pull the cut packing strap along a preset trajectory, causing it to move away from the object to be removed and to a designated recycling bin on-site. The area above the recycling bin is designated as the preset recycling location.

[0077] After the first robotic arm pulls the packing strap to the preset recycling position, the clamping component at the end of the first robotic arm opens, releasing the clamped packing strap and allowing it to fall into the recycling bin, thus completing one packing strap removal operation.

[0078] In this embodiment, after confirming that the packing strap has been cut, the second robotic arm promptly exits the cutting position to reduce the risk of the packing strap rebounding and colliding with the cutting component; the first robotic arm actively pulls and releases the cut packing strap to reduce the entanglement problem caused by the packing strap falling naturally, so that the cutting and recycling of the packing strap forms a continuous dismantling process.

[0079] In some implementations, the appropriate strapping tape recovery component is selected based on the thickness and surface condition of the strapping tape. For example, a clamping component is provided at the end of the first robotic arm to hold the cut strapping tape and pull it to a preset recovery position; another example is a suction cup provided at the end of the first robotic arm to attract the cut strapping tape and pull it to the preset recovery position. Using a suction cup to recover the strapping tape is suitable for thinner strapping tape. The strapping tape recovery component is not limited to the above examples; other components suitable for holding and pulling the cut strapping tape can be used.

[0080] This embodiment selects a clamping component or a suction cup to perform the recycling operation based on the thickness and surface condition of the strapping, providing corresponding holding and traction methods for different types of strapping and improving the adaptability of strapping recycling methods.

[0081] The following is another example, which is an illustrative example formed by combining the previous examples.

[0082] This application provides a method for removing packing straps using an embodied intelligent wheeled humanoid robot. The packing strap removal task is broken down into three functionally distinct and temporally related sub-tasks, which are completed collaboratively by two robotic arms. The first robotic arm is responsible for positioning, picking up, and traction retrieval of the straps, while the second robotic arm is responsible for precise cutting. Multimodal vision is used to recognize the position and posture of the packing straps, and an end effector force sensor is used to detect the tension of the packing straps in real time. The movement trajectory and force of the two arms are dynamically adjusted to improve the stability and success rate of the packing strap removal process.

[0083] The specific operating procedure is as follows: Global positioning and pre-preparation phase: The mobile robot navigates to the work point where the object to be dismantled is located using Simultaneous Localization and Mapping (SLAM) technology. The object to be dismantled can be a box to be unpacked. After the mobile chassis stops, the 3D camera at the end of the first robotic arm performs a global scan of the box to obtain the position and orientation of the box. By identifying the box orientation-packing strap position mapping table, the position parameters of the packing strap are determined, and the picking path of the first robotic arm and the cutting path of the second robotic arm are planned.

[0084] First robotic arm positioning and picking stage: The first robotic arm moves a color 3D camera above the packing strap to perform a high-definition scan of the packing strap. Based on the identified box pose-packing strap position mapping table, the first robotic arm feeds back more accurate packing strap position parameters. The first robotic arm then extends its electric gripper fingers at a certain angle to the gap between the packing strap and the box to correct the initial path. The first robotic arm controls the clamping jaws to close, gripping the middle position of the packing strap. A force sensor detects the tightness of the packing strap and feeds the detected value back to the controller. The controller sends a command to the first robotic arm based on the detected value to adjust the clamping force of the jaws and lift the packing strap upward. A 3D camera detects the separation distance between the packing strap and the box surface. When the separation between the packing strap and the box surface is detected to be 5-10mm, it is determined that the packing strap has been lifted to a state suitable for cutting.

[0085] The six-dimensional force sensor at the end of the first robotic arm detects the changes in force during the strapping process in real time. When a sudden change in force (slippage of the strapping) is detected, the visual detection result of the strapping is obtained. When the visual detection result indicates that the strapping has slipped, the strapping is repositioned and picked up again.

[0086] The second stage of precise cutting by the robotic arm: After the first robotic arm stabilizes the conveyor belt, it sends a cutting command to the second robotic arm. The second robotic arm moves the cutting gripper to a distance of 5mm from the cutting point. The distance between the second robotic arm and the cutting point can be detected by a sensor.

[0087] Dynamic force balance control: During the shearing process, the first robotic arm maintains a constant lifting force, and the central control sends a command to the pneumatic scissors at the end of the second robotic arm. The pneumatic scissors complete the shearing. During the shearing process, the force sensor detects the pulling force of the first robotic arm's gripper in real time. When the pulling force drops suddenly, the shearing is determined to be complete.

[0088] At the same time, the visual sensor simultaneously verifies whether the packing tape has been cut.

[0089] After the cutting is completed, the second robotic arm immediately retracts the cutting gripper to a safe position to avoid the packing strap springing back and colliding.

[0090] First robotic arm traction and recovery phase: The first robotic arm maintains its grip on the packing strap and pulls the cut strapping along a preset trajectory to the designated recycling bin on site; the gripping jaws open, and the packing strapping falls into the recycling bin, completing one packing strapping removal operation; If the box has multiple packing straps, repeat the process of the first robotic arm positioning and picking the straps, the second robotic arm precisely cutting, and the first robotic arm pulling and retrieving until all packing straps are removed.

[0091] After removing the packing straps from the current box, the robot moves to the next workstation to continue the cycle.

[0092] Based on the same inventive concept, this application also provides a system for removing packing straps from an intelligent wheeled mobile humanoid robot, comprising: a mobile composite robot body; The mobile composite robot body includes a mobile chassis, a first robotic arm, a second robotic arm, and a controller, wherein the first robotic arm and the second robotic arm are mounted on the mobile chassis; A vision sensor, located at the end of the first robotic arm, is used to acquire visual perception information of the object to be dismantled and visual detection results of the packing strap during the cutting process. A clamping assembly, disposed at the end of the first robotic arm, is used to clamp and lift the packing strap; A force sensor is installed at the end of the first robotic arm to acquire information on the first force applied by the first robotic arm during the picking process and information on the second force applied during the shearing process. A cutting component, located at the end of the second robotic arm, is used to cut the packing strap; The controller is connected to the first robotic arm, the second robotic arm, the vision sensor, the force sensor, the clamping assembly, and the shearing assembly, respectively. The controller is used to perform the following operations: The packing strap pose information is determined based on the visual perception information; Based on the packing strap pose information, determine the packing strap picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm; The first robotic arm is controlled to clamp and lift the packing strap according to the strapping operation parameters, thereby separating the packing strap from the object to be removed. The first robotic arm's operation on the packing strap is adjusted based on the first force information; While the first robotic arm holds the packing strap up, the second robotic arm is controlled to cut the packing strap using the cutting assembly according to the cutting operation parameters. Based on the change in the second force information and the visual detection result, the cutting state of the packing strap is determined.

[0093] Specifically, such as Figure 3 , Figure 4 and Figure 7 As shown, the mobile composite robot body includes a mobile chassis assembly, an upper assembly, a first robotic arm, and a second robotic arm. The upper assembly is mounted entirely on top of the mobile chassis assembly. The first and second robotic arms are respectively positioned on either side of the upper assembly to enable collaborative work between the two robotic arms. The first robotic arm can be the left robotic arm, and the second robotic arm can be the right robotic arm. The first robotic arm is responsible for identifying, gripping, and picking up the packing straps, while the second robotic arm is responsible for cutting the packing straps.

[0094] It should be noted that the mounting sides of the first and second robotic arms can be interchanged. That is, the right robotic arm performs the identification, clamping, lifting and traction recovery of the packing straps, while the left robotic arm performs the cutting of the packing straps.

[0095] like Figure 5 As shown, the end effector of the first robotic arm is equipped with a vision sensor 1, a gripping assembly 2, and a force sensor 3. The vision sensor 1 can be a 3D camera, such as a color 3D camera, used to acquire visual perception information of the object to be removed and the packing strap, and to acquire visual inspection results of the packing strap during the cutting process; the gripping assembly 2 can be an electric gripper, which includes a gripper and a contoured finger. The surface of the contoured finger is coated with rubber to reduce damage to the surface of the object to be removed during gripping; the force sensor 3 is directly connected to the end effector of the first robotic arm and is used to detect the force exerted by the first robotic arm during the gripping, lifting, and cutting processes.

[0096] In some implementations, the vision sensor 1 can be replaced by a line laser camera to perform edge detection on the packing strap.

[0097] like Figure 6 As shown, the end of the second robotic arm is equipped with a shearing component, which can be a pneumatic shear. The pneumatic shear is connected to the end of the second robotic arm via an adapter flange and is used to cut the packing strap in the picking state. The first and second robotic arms can be a six-axis cooperative robotic arm to drive the clamping component and the shearing component to the corresponding working positions, respectively.

[0098] The controller is connected to the first robotic arm, the second robotic arm, vision sensor 1, gripping assembly 2, force sensor 3, and shearing assembly. After acquiring visual perception information of the object to be removed, vision sensor 1 transmits this information to the controller. The controller determines the position and orientation of the packing strap based on the visual perception information, obtaining the packing strap pose information. Based on this pose information, the controller then determines the packing strap picking parameters for the first robotic arm and the shearing parameters for the second robotic arm.

[0099] The controller, based on the strapping operation parameters, directs the first robotic arm to bring the clamping assembly 2 close to the strapping, controls the clamping assembly 2 to close and clamp the strapping, and then controls the first robotic arm to lift the strapping upwards, separating the strapping from the object to be removed. During the strapping process, the force sensor 3 detects the force acting on the first robotic arm and obtains the first force information; the controller adjusts the strapping state of the first robotic arm based on the first force information, ensuring that the first robotic arm maintains clamping and lifting of the strapping.

[0100] While the first robotic arm maintains its carrying position, the controller, based on the shearing operation parameters, controls the second robotic arm to move the shearing assembly to the shearing position, thus shearing the packing strap. During the shearing process, force sensor 3 detects the force acting on the first robotic arm, obtaining second force information; vision sensor 1 simultaneously acquires the visual detection results of the packing strap. The controller determines whether the packing strap is cut or not based on the changes in the second force information and the visual detection results.

[0101] By placing a vision sensor, a gripping assembly, and a force sensor at the end of the first robotic arm, and a shearing assembly at the end of the second robotic arm, a dual-arm collaborative structure is formed, in which the first robotic arm is responsible for identification and picking up the strap, and the second robotic arm is responsible for shearing. The picking state is adjusted by utilizing the force information during the picking process, and the shearing state is determined by combining the changes in force information during the shearing process and the visual detection results. This improves the stability and reliability of the picking, shearing, and shearing judgment processes of flexible packing straps.

[0102] In some embodiments, the mobile chassis assembly employs a differential drive method and is equipped with LiDAR and safety contact edges to enable autonomous navigation, obstacle avoidance, and safety protection for the mobile composite robot. The mobile chassis assembly has an internal mounting cavity for securing the central control unit and power supply unit. Driven wheels and drive wheels are located at the bottom of the mobile chassis assembly, which propel the mobile composite robot. The outer shell of the mobile chassis assembly has a charging port, debugging port, buttons, and multiple heat dissipation holes. Two radars are located on the top of the mobile chassis assembly to assist the mobile composite robot in navigation.

[0103] The mobile chassis, equipped with differential drive, LiDAR, and safety contact edges, provides navigation, obstacle avoidance, and safety protection for the mobile composite robot to move between different work positions and approach objects to be demolished. The central control unit and power supply unit are provided with installation, debugging, power supply, and heat dissipation conditions through the mounting cavity, charging port, debugging port, and heat dissipation holes.

[0104] In some embodiments, the superstructure includes a superstructure shell assembly, a superstructure internal structure assembly, a first robotic arm, a second robotic arm, and an upper body shell assembly. Both the first and second robotic arms are six-axis collaborative robotic arms, and the load capacity of a single robotic arm can be 10 kg. The upper body shell assembly is formed from ABS material through 3D printing and consists of two shell parts, front and rear, to reduce the weight of the superstructure assembly. A head assembly may also be mounted on the top of the mobile composite robot body.

[0105] The first and second robotic arms are mounted on the mobile chassis by an upper assembly, providing independent yet collaborative working spaces for the two robotic arms. The use of a lightweight upper shell assembly helps reduce the load on the mobile chassis and provides protection for the internal structure of the mobile composite robot.

[0106] In this specification, the same or similar parts between the various embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the descriptions of the embodiments described later are relatively simple, and relevant parts can be referred to the descriptions of the foregoing embodiments.

[0107] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for removing packing straps using an embodied intelligent wheeled humanoid robot, characterized in that, include: Obtain visual perception information of the object to be demolished; The packing strap pose information is determined based on the visual perception information; Based on the packing strap pose information, determine the packing strap picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm; The first robotic arm is controlled to lift the packing strap according to the picking operation parameters, so that the packing strap is separated from the object to be removed; Acquire the first force information of the first robotic arm during the lifting process, and adjust the operation of the first robotic arm on the packing strap according to the first force information; While the first robotic arm holds the packing strap up, the second robotic arm is controlled to cut the packing strap according to the cutting operation parameters; The second force information of the first robotic arm during the cutting process and the visual detection results of the packing strap during the cutting process are obtained to determine the cutting state of the packing strap.

2. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 1, characterized in that, The step of determining the strapping operation parameters of the first robotic arm and the shearing operation parameters of the second robotic arm based on the strapping pose information includes: The physical attribute information of the packing tape is determined based on the visual perception information, wherein the physical attribute information includes at least one of color and bandwidth; Based on the packing strap pose information and the physical attribute information, the picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm are determined.

3. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 1, characterized in that, The method further includes: Based on the visual perception information, determine the spatial topology information of the packing tape; Based on the spatial topology information, determine the bundling pattern of the packing strap; When multiple packing straps exist on the object to be dismantled, the dismantling priority among the multiple packing straps is determined according to the binding pattern; The multiple packing straps were removed according to the removal priority.

4. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 1, characterized in that, The operation of adjusting the first robotic arm's force on the packing strap based on the first force information includes: The tightness of the packing strap is determined based on the first force information; The clamping force of the first robotic arm on the packing strap is adjusted according to the tightness to maintain the first robotic arm's picking state of the packing strap.

5. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 4, characterized in that, The method further includes: The force changes during the lifting process are determined based on the first force information; When the force change represents a sudden change in force, the visual detection results of the packing strap during the strap picking process are obtained; When it is determined that the packing strap has slipped based on the visual detection results during the packing strap picking process, the packing strap pose information is re-determined based on the visual detection results during the packing strap picking process. Based on the redefined strapping pose information, the strapping operation parameters are updated, and the first robotic arm is controlled to clamp and lift the strapping again according to the updated strapping operation parameters.

6. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 1, characterized in that, The determination of the cut state of the packing strap includes: Based on the second force information, determine the change in force acting on the first robotic arm; When the force change characterizes the sudden drop in the force on the first robotic arm, an initial cutting determination result characterizing that the packing strap has been cut is obtained; Based on the visual inspection results, the initial cutting determination results are verified to determine the cutting state of the packing strap.

7. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 1, characterized in that, The method further includes: Based on the visual perception information, determine the size information of the object to be demolished; Perform at least one of the following operations based on the size information: Determine the end-effector approach postures of the first and second robotic arms; Adjust the workspace boundaries of the first robotic arm and the second robotic arm.

8. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 1, characterized in that, The step of acquiring visual perception information of the object to be dismantled, and determining the packing strap pose information based on the visual perception information, includes: A global scan of the object to be demolished is performed to obtain global visual perception information; Based on the global visual perception information, determine the pose information of the object to be demolished; Based on the pose information of the object to be dismantled and the preset pose mapping relationship, the initial position information of the packing strap is determined. The pose mapping relationship is used to characterize the correspondence between the pose of the object to be dismantled and the initial position of the packing strap. The local scanning area is determined based on the initial position information, and the local scanning area is scanned to obtain the local visual perception information of the packing tape. The packing strap pose information is determined based on the local visual perception information.

9. The method for removing packing straps from an intelligent wheeled humanoid robot according to claim 1, characterized in that, The method further includes: After determining that the packing strap is cut, the second robotic arm is controlled to exit the cutting position; The first robotic arm is controlled to maintain its grip on the packing strap and to pull the cut packing strap to a preset recycling position along a preset trajectory. Control the first robotic arm to release the packing strap.

10. A system for removing packing straps from a humanoid wheeled robot with intelligent features, characterized in that: include: Mobile composite robot body; The mobile composite robot body includes a mobile chassis, a first robotic arm, a second robotic arm, and a controller, wherein the first robotic arm and the second robotic arm are mounted on the mobile chassis; A vision sensor, located at the end of the first robotic arm, is used to acquire visual perception information of the object to be dismantled and visual detection results of the packing strap during the cutting process. A clamping assembly, disposed at the end of the first robotic arm, is used to clamp and lift the packing strap; A force sensor is installed at the end of the first robotic arm to acquire information on the first force applied by the first robotic arm during the picking process and information on the second force applied during the shearing process. A cutting component, located at the end of the second robotic arm, is used to cut the packing strap; The controller is connected to the first robotic arm, the second robotic arm, the vision sensor, the force sensor, the clamping assembly, and the shearing assembly, respectively. The controller is used to perform the following operations: The packing strap pose information is determined based on the visual perception information; Based on the packing strap pose information, determine the packing strap picking operation parameters of the first robotic arm and the cutting operation parameters of the second robotic arm; The first robotic arm is controlled to clamp and lift the packing strap according to the strapping operation parameters, thereby separating the packing strap from the object to be removed. The first robotic arm's operation on the packing strap is adjusted based on the first force information; While the first robotic arm holds the packing strap up, the second robotic arm is controlled to cut the packing strap using the cutting assembly according to the cutting operation parameters. Based on the change in the second force information and the visual detection result, the cutting state of the packing strap is determined.

Images