Humanoid robot control method, system, device, medium, and program

Abstract

The invention relates to the technical field of robot control, and provides a humanoid robot control method, system, device, medium and program, the control method comprises the following steps: controlling an environment sensing unit to scan an operation area and generate first environment point cloud data representing a static map; controlling a visual perception unit to acquire pose data of the target object; a force sense feedback unit integrated at the tail end of the mechanical arm executing mechanism is controlled, and force sense feedback data of the mechanical arm executing mechanism in the process of making contact with the target object is detected; according to the pose data, the first environment point cloud data and a preset operation strategy library, a motion track of a mechanical arm executing mechanism is generated; controlling an environment sensing unit to scan the operation area in real time and generate second environment point cloud data representing dynamic obstacle distribution; and correcting the motion track in real time according to the second environment point cloud data to obtain an obstacle avoidance path. The method is used for solving the problems of safe operation and high-precision operation of the humanoid robot in a dynamic change environment.

Description

Technical Field

[0001] This invention relates to the field of robot control technology, and in particular to a control method, system, device, medium and program for a humanoid robot. Background Technology

[0002] Among related technologies, humanoid robots, as an important carrier of embodied intelligence, have broad application prospects in unstructured scenarios such as industrial assembly and logistics handling, including depalletizing and palletizing tasks. In these industrial applications, robots need to autonomously complete the entire process from environmental perception and path planning to final execution. However, existing control technologies still have significant limitations in practical applications.

[0003] Firstly, in terms of motion planning and obstacle avoidance, traditional methods often separate static environment modeling from dynamic obstacle detection. Most existing technologies only use LiDAR or depth cameras to build a static map once during the task initialization phase to generate an initial trajectory. However, during the robot's motion, there is a lack of a continuous real-time scanning mechanism for the environment; or although they have dynamic detection capabilities, they fail to deeply integrate and instantly correct the real-time generated dynamic point cloud data with the initial motion trajectory. This "open-loop" or "semi-closed-loop" control mode often leads to collisions when the robot encounters suddenly appearing moving obstacles due to perception lag or untimely replanning, making it difficult to guarantee motion safety in complex dynamic environments.

[0004] Secondly, in the interaction between the robotic arm and the target object, existing technologies mostly rely on pure position control or simple visual servo positioning. Due to errors in visual perception, limitations in the repeatability accuracy of the robotic arm itself, and the possibility of slight pose deviations in the target object, pure position control is prone to generating rigid impacts at the moment the robotic arm's end effector contacts the object. This impact force can not only damage the delicate target object or the robotic arm's end effector, but may also lead to grasping failure or the object slipping. Existing control schemes generally lack mechanisms for real-time acquisition of force feedback data during contact and dynamic adjustment of the operating force accordingly, failing to achieve "compliant" interaction similar to humans and limiting the application capabilities of humanoid robots in delicate tasks such as high-precision assembly or handling of fragile items. During the robotic arm's pose movement, there is often a lack of real-time judgment and response mechanisms for abnormal states such as target object position deviation, sudden appearance of environmental obstacles, and obstruction of the robotic arm, which can easily lead to operation failure or equipment damage due to unexpected situations.

[0005] Therefore, there is an urgent need for a control method, system, device, medium, and program for humanoid robots to improve the above-mentioned problems. Summary of the Invention

[0006] This invention provides a control method, system, device, medium, and program for a humanoid robot, which is used to solve the problem of safe operation and high-precision operation of humanoid robots in dynamically changing environments.

[0007] According to a first aspect of the present invention, a control method for a humanoid robot is provided, applied to a humanoid robot system including a visual perception unit, an environmental perception unit, a robotic arm actuator, and a force feedback unit. The control method includes: controlling the environmental perception unit to scan a work area and generate first environmental point cloud data representing a static map; controlling the visual perception unit to acquire pose data of a target object; controlling the force feedback unit integrated at the end of the robotic arm actuator to detect force feedback data during contact with the target object; determining whether an abnormal state is triggered based on the pose data and the force feedback data; when no abnormal state is triggered, identifying the arrangement type of target material frames in the work area based on the pose data; and matching the arrangement type of the material frames from a preset work strategy library. The depalletizing or palletizing strategy is used as the current execution strategy. When an abnormal state is triggered, the abnormal handling strategy is selected as the current execution strategy. Based on the current execution strategy, combined with the spatial coordinates and attitude angle of the target material box in the pose data, and the first environmental point cloud data, the end effector path and joint motion sequence of the robotic arm are calculated to generate the motion trajectory of the robotic arm actuator. During the process of driving the robotic arm actuator to perform pose motion, the control environment perception unit scans the working area in real time and generates second environmental point cloud data representing the distribution of dynamic obstacles. The motion trajectory is corrected in real time according to the second environmental point cloud data to obtain the obstacle avoidance path. The robotic arm actuator is controlled to continue to perform pose motion along the real-time obstacle avoidance path. The operating force of the robotic arm actuator relative to the target object is dynamically adjusted according to the force feedback data.

[0008] In one embodiment, the target object includes a target frame and a target placement position, and the pose data includes a target grasping pose and a target placement pose. Controlling the visual perception unit to acquire the pose data of the target object includes: after the humanoid robot moves to the target work area, controlling the visual perception unit to scan the target object and acquire original point cloud data containing the target frame and the target placement position; segmenting the original point cloud data to separate a first point cloud cluster corresponding to the target frame and a second point cloud cluster corresponding to the target placement position; based on the first point cloud cluster, identifying the grasping surface and grasping points of the target frame, and calculating the target grasping pose of the target frame in the robot's base coordinate system; and based on the second point cloud cluster, calculating the target placement pose of the target placement position in the robot's base coordinate system.

[0009] In one embodiment, during the process of driving the robotic arm actuator to perform pose movement, the environmental perception unit controls the real-time scanning of the work area and generates second environmental point cloud data representing the distribution of dynamic obstacles. This includes: driving the humanoid robot to move to the work area based on a static map; after arriving at the work area, controlling the environmental perception unit to perform a three-dimensional scan of the current work area to obtain first environmental point cloud data containing surrounding obstacles; comparing and analyzing the first environmental point cloud data with the static map, removing background points and dynamic interference points that overlap with the static map, extracting obstacle features that have changed or been added relative to the static map, and generating second environmental point cloud data.

[0010] In one implementation, a preset operation strategy library includes depalletizing strategies, palletizing strategies, and anomaly handling strategies. Generating the motion trajectory of the robotic arm actuator includes: determining whether an abnormal state is triggered based on pose data and force feedback data; when no abnormal state is triggered, identifying the arrangement type of the target material frame in the operation area based on the pose data; matching the depalletizing strategy or palletizing strategy corresponding to the material frame arrangement type from the operation strategy library as the current execution strategy; when an abnormal state is triggered, selecting the anomaly handling strategy as the current execution strategy; and based on the current execution strategy, combined with the spatial coordinates and attitude angles of the target material frame in the pose data, and the first environmental point cloud data, calculating the end effector path and joint motion sequence of the robotic arm to generate the motion trajectory of the robotic arm actuator.

[0011] In one implementation, determining whether an abnormal state is triggered based on pose data and force feedback data includes: real-time detection of force feedback data; when the sudden change in the operating force of the robotic arm actuator relative to the target object exceeds a preset force threshold, determining that a collision or grasp slippage has occurred, triggering a first abnormal state; and real-time detection of pose data; when the tracking features of the target object are detected to be lost, triggering a second abnormal state.

[0012] In one embodiment, the robotic arm actuator includes a first robotic arm and a second robotic arm; the destacking strategy includes: when it is detected that the distance between the target frame and the adjacent frame is less than or equal to a safety threshold, controlling the first robotic arm to grasp the first side of the target frame and pull the target frame to a preset separation distance in a direction away from the adjacent frame; controlling the second robotic arm to grasp the second side of the target frame, wherein the first side and the second side are opposite sides of the target frame; when it is confirmed that the distance is greater than the safety threshold, controlling the first robotic arm and the second robotic arm to work together to remove the target frame from the pallet.

[0013] In one embodiment, the robotic arm actuator includes a first robotic arm and a second robotic arm; the palletizing strategy includes: identifying the current working scenario based on pose data; when the current working scenario is an initial empty pallet, determining the palletizing target pose based on the pose of the empty pallet; when the current working scenario is an existing pallet type, determining the palletizing target pose based on the pose of the top layer frame; when performing the palletizing operation of the same layer of frames, calculating the expected distance between the target frame and the already placed frames; when the expected distance is greater than or equal to a safety threshold, controlling the first robotic arm and the second robotic arm to cooperate in placing the target frame in the target pose and simultaneously unloading the target frame; when the expected distance is less than the safety threshold, controlling the first robotic arm and the second robotic arm to cooperate in transporting the target frame to a preset position, controlling the gripper on the side close to the already placed adjacent frame to release, and the other gripper to hold the target frame alone; controlling the robotic arm on the holding side to move in the direction toward the already placed adjacent frame, pushing the target frame until it contacts the already placed adjacent frame.

[0014] In one implementation, the anomaly handling strategy includes: when a first abnormal state is triggered, controlling the robotic arm actuator to stop its current motion trajectory and maintain its current position for a preset time; controlling the robotic arm actuator to retreat a preset safe distance along the direction of the sudden change in operating force; after the retreat is completed, replanning the path for grabbing or placing the target material frame and performing a compensation operation; if the number of consecutive compensation operations exceeds a preset retry threshold, triggering an emergency stop alarm; when a second abnormal state is triggered, controlling the robotic arm actuator to pause operation and move to a preset safe waiting posture; if the target material frame is relocked within a preset time window, updating the motion plan based on the newly acquired posture data and resuming operation; if the target material frame is not locked within the time limit, triggering a manual intervention alarm; when it is detected in real time that both the first and second abnormal states have disappeared, controlling the robotic arm actuator to continue performing subsequent actions along the motion trajectory.

[0015] In one implementation, the motion trajectory is corrected in real time based on the second environmental point cloud data to obtain an obstacle avoidance path, including: constructing a local cost map containing dynamic obstacles in real time based on the second environmental point cloud data, the dynamic obstacles including moving workers or target objects whose positions change; mapping the motion trajectory to the local cost map for collision detection; if a conflict is detected between the motion trajectory and the dynamic obstacle, the joint angle sequence of the robotic arm actuator is replanned online in the joint space to generate an obstacle avoidance path.

[0016] In one embodiment, dynamically adjusting the operating force of the robotic arm actuator relative to the target object based on force feedback data includes: during the grasping phase, controlling the robotic arm actuator to move to a preset coarse positioning pose based on visual servoing; then switching to a constant force admittance control mode, calculating the pose correction amount and adjusting the end-effector pose of the robotic arm actuator based on the contact force deviation collected in real time by the force feedback unit, until a fine positioning pose is reached and the grasping of the target object is completed; during the placement phase, controlling the robotic arm actuator to move to a preset coarse positioning pose based on visual servoing; then switching to a constant force admittance control mode, calculating the pose correction amount and adjusting the end-effector pose of the robotic arm actuator based on the contact force deviation collected in real time by the force feedback unit, so that the target object approaches the target placement surface with a constant contact force, until the placement of the target object is completed.

[0017] According to a second aspect of the present invention, a humanoid robot system is provided for performing the method of any one of the first aspects. The system includes: an environment perception unit for scanning a work area and generating first environmental point cloud data representing a static map; a vision perception unit for acquiring pose data of a target object; a force feedback unit for detecting force feedback data of a robotic arm actuator during contact with the target object; the force feedback unit is integrated into the end effector of the robotic arm actuator; and a central control unit for determining whether an abnormal state is triggered based on the pose data and the force feedback data; when no abnormal state is triggered, identifying the arrangement type of the target material frame in the work area based on the pose data; matching a destacking strategy or palletizing strategy corresponding to the material frame arrangement type from a preset work strategy library as the current execution strategy; when an abnormal state is triggered, selecting an abnormal handling strategy as the current execution strategy; and based on the current execution strategy, combining the target object in the pose data... The spatial coordinates and attitude angles of the target frame, along with the first environmental point cloud data, are used to calculate the end effector path and joint motion sequence of the robotic arm, generating the motion trajectory of the robotic arm actuator. During the pose motion of the robotic arm actuator, the environmental perception unit scans the work area in real time and generates second environmental point cloud data representing the distribution of dynamic obstacles. The motion trajectory is corrected in real time based on the second environmental point cloud data to obtain an obstacle avoidance path. The chassis of the humanoid robot is controlled according to a static map to move the humanoid robot to the work area. A waist unit connects the chassis and the robotic arm actuator. The waist unit drives the lifting and pitch angles of the robotic arm actuator. The robotic arm actuator continues to perform pose motion along the real-time obstacle avoidance path. The operating force of the robotic arm actuator relative to the target object is dynamically adjusted based on force feedback data, and the pose correction amount is calculated and the end effector pose of the robotic arm actuator is adjusted based on the force feedback data.

[0018] According to a third aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method described above.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that the above-described method can be implemented when the executable computer program in the storage medium is executed by a processor.

[0020] According to a fifth aspect of the present invention, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described above.

[0021] Compared with existing technologies, the advantages of this invention are as follows: By controlling the environmental perception unit to scan the work area and generate first environmental point cloud data representing a static map and controlling the visual perception unit to collect the pose data of the target object, and combining this with a preset work strategy library to generate the motion trajectory of the robotic arm actuator, a collision-free initial planning based on global information is achieved. Furthermore, by scanning the work area in real time and generating second environmental point cloud data representing the distribution of dynamic obstacles during the movement, the motion trajectory is corrected in real time to obtain an obstacle avoidance path, and the pose movement is continued along the real-time obstacle avoidance path, upgrading open-loop tracking to dynamic closed-loop obstacle avoidance, effectively avoiding the collision risk caused by sudden interference. Finally, by detecting the force feedback data of the robotic arm actuator during contact with the target object and dynamically adjusting the operating force of the robotic arm actuator relative to the target object based on the force feedback data, rigid contact impact is eliminated, realizing full-process adaptive control from macroscopic dynamic obstacle avoidance to microscopic compliant operation, improving the safety and accuracy of the operation. This invention also introduces an abnormal state judgment mechanism based on pose and force data to achieve intelligent dynamic switching of operation strategies: Under normal conditions, the system can accurately identify the material frame arrangement type and match the optimal destacking and stacking strategies to ensure operation efficiency; under abnormal conditions, the system can switch to the abnormal handling strategy in a timely manner to effectively deal with emergencies and avoid equipment damage or operation failure. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating a control method, system, device, medium, and program method for a humanoid robot according to an exemplary embodiment.

[0023] Figure 2 This is a schematic diagram illustrating a palletizing distribution according to an exemplary embodiment.

[0024] Figure 3This is a structural block diagram of a humanoid robot system according to another exemplary embodiment.

[0025] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0026] Explanation of the reference numerals in the figure:

[0027] 1. Humanoid robot system; 2. Environmental perception unit; 3. Visual perception unit; 4. Force feedback unit; 5. Central control unit; 6. Robotic arm actuator; 7. Pallet; 8. Target material frame; 9. Adjacent material frames; 10. Waist unit; 11. Chassis; 900. Electronic device; 922. Processing component; 926. Power supply component; 932. Memory; 950. Network interface; 958. Input / output interface. Detailed Implementation

[0028] Unless otherwise defined, the technical or scientific terms used in this specification should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. Specific embodiments of the invention will be described below with reference to the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot provide a detailed description of all features of the actual embodiments. Without departing from the spirit and scope of the invention, those skilled in the art can make modifications and substitutions to the embodiments of the invention, and the resulting embodiments are also within the protection scope of the invention.

[0029] like Figure 1 As shown, the first embodiment of the present invention provides a control method for a humanoid robot, applied to a humanoid robot system including a visual perception unit, an environmental perception unit, a robotic arm actuator, and a force feedback unit. The control method includes the following steps S1-S6: S1 controls the environmental perception unit to scan the work area and generate the first environmental point cloud data representing a static map.

[0030] S2 controls the visual perception unit to acquire the pose data of the target object.

[0031] S3 controls the force feedback unit integrated at the end of the robotic arm actuator to detect force feedback data during the process of the robotic arm actuator contacting the target object.

[0032] S4, determine whether an abnormal state is triggered based on the pose data and the force feedback data; when no abnormal state is triggered, identify the arrangement type of the target material frame in the work area based on the pose data; match the destacking strategy or palletizing strategy corresponding to the material frame arrangement type from the preset work strategy library as the current execution strategy; when an abnormal state is triggered, select the abnormal handling strategy as the current execution strategy; based on the current execution strategy, combined with the spatial coordinates and posture angle of the target material frame in the pose data, and the first environmental point cloud data, calculate the end effector path and joint motion sequence of the robotic arm, and generate the motion trajectory of the robotic arm actuator.

[0033] S5, during the process of driving the robotic arm actuator to perform pose movement, the control environment perception unit scans the working area in real time and generates second environmental point cloud data representing the distribution of dynamic obstacles; the motion trajectory is corrected in real time based on the second environmental point cloud data to obtain the obstacle avoidance path; the control robotic arm actuator continues to perform pose movement along the real-time obstacle avoidance path.

[0034] S6 dynamically adjusts the operating force of the robotic arm actuator relative to the target object based on force feedback data.

[0035] like Figure 2 As shown, in one embodiment, the target object includes a target frame 8 and a target placement position, and the pose data includes a target grasping pose and a target placement pose. Controlling the visual perception unit 3 to acquire the pose data of the target object includes: after the humanoid robot moves to the target work area, controlling the visual perception unit 3 to scan the target object and acquire original point cloud data containing the target frame 8 and the target placement position; segmenting the original point cloud data to separate a first point cloud cluster corresponding to the target frame 8 and a second point cloud cluster corresponding to the target placement position; based on the first point cloud cluster, identifying the grasping surface and grasping points of the target frame 8, and calculating the target grasping pose of the target frame 8 in the robot's base coordinate system; and based on the second point cloud cluster, calculating the target placement pose of the target placement position in the robot's base coordinate system.

[0036] In one specific implementation, the target object encompasses the target material frame 8 to be manipulated and a preset target placement position. The corresponding pose data is specifically refined into the target grasping pose and the target placement pose. After the humanoid robot moves to the designated target work area according to global planning, the control system immediately instructs the vision perception unit 3 to perform a high-resolution scan of the work scene to acquire raw 3D point cloud data containing information about the target material frame 8, the target placement position, and the surrounding environment. Subsequently, the data processing module performs preprocessing and segmentation algorithms on the raw point cloud data: first, noise is removed by filtering; then, using geometric features or a deep learning semantic segmentation network, the raw data is accurately separated into a first point cloud cluster corresponding to the target material frame 8 and a second point cloud cluster corresponding to the target placement position. For the first point cloud cluster, the system further extracts its geometric topological features, identifies the optimal grasping surface and specific grasping point coordinates that meet the gripper opening and closing requirements, and calculates the six-degree-of-freedom target grasping pose of the material frame in the robot's base coordinate system by combining the preset approach direction vector. Simultaneously, for the second cloud cluster, the system analyzes its plane fitting parameters or feature matching results to determine the center point and normal direction of the target placement area, thereby calculating the precise target placement pose. Through the above step-by-step processing, the robot can simultaneously establish a complete spatial mapping relationship of "where to grab" and "where to place" in the base coordinate system, providing high-precision data support for the subsequent generation of collision-free picking and placing motion trajectories, effectively overcoming the positioning failure problem caused by arbitrary placement of the material frame or slight deviations in the placement area.

[0037] In one embodiment, during the pose movement of the robotic arm actuator 6, the environmental perception unit 2 is controlled to scan the work area in real time and generate second environmental point cloud data representing the distribution of dynamic obstacles. This includes: driving the humanoid robot to move to the work area based on a static map; after arriving at the work area, controlling the environmental perception unit 2 to perform a three-dimensional scan of the current work area to obtain first environmental point cloud data containing surrounding obstacles; comparing and analyzing the first environmental point cloud data with the static map, removing background points and dynamic interference points that overlap with the static map, extracting obstacle features that have changed or been added relative to the static map, and generating second environmental point cloud data.

[0038] In one specific implementation, during the pose movement of the robotic arm actuator 6, the system employs a strategy combining static reference guidance and dynamic real-time differential mapping to control the environment perception unit 2 to scan the work area and generate second environmental point cloud data representing the distribution of dynamic obstacles. Specifically, the robot first plans and drives the robotic arm actuator 6 to move along the initial trajectory to the target work area based on the high-precision static map constructed by the environment perception unit 2 during the task initialization phase. This static map serves as a reference model for the environmental background and pre-stores the geometric information of invariant elements such as fixed facilities, walls, and stationary workbenches.

[0039] When the robot arrives at or approaches the work area, the control system immediately triggers the environmental perception unit 2 to perform a high-frequency 3D scan of the current local space, acquiring first environmental point cloud data containing all surrounding objects. Subsequently, the data processing module executes rigorous point cloud registration and differential analysis algorithms: the real-time acquired first environmental point cloud data is spatially aligned with a pre-stored static map in a unified coordinate system. Through point-by-point comparison or voxel mesh matching technology, point cloud data that highly overlaps with background features in the static map is automatically removed, while dynamic interference points caused by sensor noise are filtered out. After this processing, the system retains only point cloud clusters that have shifted position, changed shape, or newly appeared relative to the static map. These remaining data are extracted and reconstructed into second environmental point cloud data. This second environmental point cloud data accurately represents the real-time distribution of dynamic obstacles within the work area, providing low-latency, high signal-to-noise ratio perceptual input for subsequent trajectory replanning, ensuring that the robot can promptly identify risks and take obstacle avoidance actions in complex dynamic environments.

[0040] In one implementation, a preset operation strategy library includes depalletizing strategies, palletizing strategies, and anomaly handling strategies. Generating the motion trajectory of the robotic arm actuator 6 includes: determining whether an abnormal state is triggered based on pose data and force feedback data; when no abnormal state is triggered, identifying the arrangement type of the target material frame 8 in the operation area based on the pose data; matching the depalletizing strategy or palletizing strategy corresponding to the material frame arrangement type from the operation strategy library as the current execution strategy; when an abnormal state is triggered, selecting the anomaly handling strategy as the current execution strategy; and based on the current execution strategy, combining the spatial coordinates and attitude angles of the target material frame 8 in the pose data, and the first environmental point cloud data, calculating the end effector path and joint motion sequence of the robotic arm to generate the motion trajectory of the robotic arm actuator.

[0041] In one specific implementation, a pre-defined operation strategy library constructs a strategy set including standardized depalletizing strategies, standardized palletizing strategies, and multi-dimensional anomaly handling strategies. When generating the motion trajectory of the robotic arm actuator 6, the system first executes real-time status monitoring logic: it integrates the real-time pose data provided by the visual perception unit 3 with the contact force / torque data collected by the force feedback unit 4, and determines whether the current operation process triggers abnormal states such as jamming, collision, grasp slippage, or target loss by setting threshold comparison or trend analysis algorithms.

[0042] If no abnormal state is triggered, the system enters the normal operation process: First, based on the pose data, the spatial distribution of the target material frames 8 in the operation area is analyzed to identify their arrangement type; then, the system searches and matches the depalletizing or palletizing strategy that best suits the specific arrangement type in the operation strategy library and establishes it as the current execution strategy. This strategy predefines the standard grasping order, placement path planning logic, and speed acceleration constraints for this type of arrangement.

[0043] If an abnormal state is detected, the system immediately interrupts the normal process and selects the corresponding abnormal handling strategy from the job strategy library as the current execution strategy. Such strategies include, but are not limited to: performing micro-jitter to release the jam, retracting along the normal direction to avoid collision, replanning the grab point to deal with slippage, or moving to a safe waiting area and sending an alarm signal.

[0044] After determining the current execution strategy, the trajectory generation module deeply integrates the logical rules of the strategy with multi-source perception data: using the precise spatial coordinates (x, y, z) and attitude angles (roll, pitch, yaw) of the target bounding box 8 in the pose data to determine the start and end points, and combining the real-time static obstacle distribution represented by the first environmental point cloud data, a sampling-based planning algorithm, such as Rapidly-exploring Random Tree Star (RRT), is adopted. Alternatively, optimization-based planning algorithms, such as Covariant Hamiltonian Optimization for Motion Planning (CHOMP) or Stochastic Trajectory Optimization for Motion Planning (STOMP), can be used to calculate a collision-free Cartesian space path for the robotic arm's end effector that satisfies dynamic constraints. This path is then converted into a motion sequence for each joint through inverse kinematics calculations. Finally, a smooth, safe, and efficient complete motion trajectory command is generated and sent to the underlying controller for execution. This motion sequence includes curves showing the changes in joint angles, angular velocities, and angular acceleration over time.

[0045] In one implementation, determining whether an abnormal state is triggered based on pose data and force feedback data includes: real-time detection of force feedback data; when the operating force of the robotic arm actuator 6 relative to the target object suddenly exceeds a preset force threshold, determining that a collision or grasp slippage has occurred, triggering a first abnormal state; and real-time detection of pose data; when the tracking features of the target object are detected to be lost, triggering a second abnormal state.

[0046] In one specific implementation, a dual real-time monitoring mechanism based on multimodal sensor fusion is constructed to determine whether an abnormal state is triggered based on pose data and force feedback data. First, the system collects and analyzes six-dimensional force / torque sensor data installed on the robotic arm wrist or end effector at a high frequency, for example, 1 kHz, forming a continuous force feedback time sequence. The control algorithm calculates the rate of change and absolute value of the operational force in real time within this sequence. Once it detects a sudden change in contact force or torque within a very short time during the robotic arm's grasping, lifting, or placing actions, and the magnitude of this change exceeds a preset dynamic force threshold, the system determines that an unexpected rigid collision or slippage of the target object has occurred, thus triggering the first abnormal state.

[0047] Simultaneously, the system runs a parallel visual tracking and monitoring module, processing the pose data stream output by the visual perception unit 3 in real time. This module continuously locks onto key geometric feature points or artificial markers of the target object, such as QR codes or specific edge contours, and calculates its tracking confidence and positional continuity across consecutive frames. When the tracking features of the target object are lost due to the object moving rapidly out of the camera's field of view, being completely occluded by a sudden obstacle, or the object's surface reflection / texture causing feature extraction failure, the system determines that visual guidance has failed and immediately triggers a second abnormal state. Through this dual redundancy verification of force and vision, the robot can accurately identify different types of operational faults within milliseconds, providing a solid triggering basis for quickly switching to anomaly handling strategies and effectively preventing downtime caused by false alarms from a single sensor or equipment damage caused by missed alarms.

[0048] In one embodiment, the robotic arm actuator 6 includes a first robotic arm and a second robotic arm; the destacking strategy includes: when it is detected that the distance between the target frame 8 and the adjacent frame 9 is less than or equal to a safety threshold, controlling the first robotic arm to grasp the first side of the target frame 8 and pull the target frame 8 to a preset separation distance in a direction away from the adjacent frame 9, and controlling the second robotic arm to grasp the second side of the target frame 8, wherein the first side and the second side are opposite sides of the target frame 8; when it is confirmed that the distance is greater than the safety threshold, controlling the first robotic arm and the second robotic arm to work together to remove the target frame 8 from the pallet 7.

[0049] In one specific implementation, the robotic arm actuator 6 is configured as a dual-arm collaborative operation system comprising a first robotic arm and a second robotic arm to handle destacking tasks in high-density stacking scenarios. The destacking strategy is specifically optimized for situations where the distance between the target material frame 8 and its adjacent material frames 9 is too small: when the visual perception unit 3 identifies that the gap between the target material frame 8 and its surrounding adjacent material frames 9 is less than or equal to a preset safety threshold, the system automatically initiates a two-stage collaborative process of "separation before removal".

[0050] During the separation phase, the control system first instructs the first robotic arm to move to the first side of the target frame 8, using its end effector to firmly grasp the edge or a dedicated handle on that side. Subsequently, the first robotic arm applies a controlled traction force along the normal direction away from the adjacent frame 9, horizontally moving the target frame 8 to a preset separation distance. This action aims to artificially increase the gap between the target frame 8 and surrounding obstacles, eliminating the risk of interference caused by stacking errors or cargo deformation, and creating sufficient operating space for the intervention of the second robotic arm.

[0051] Once the system confirms that the current gap is significantly greater than the safety threshold, it enters the collaborative removal phase. At this time, the control system instructs the second robotic arm to move rapidly and grasp the second side of the target material frame 8, forming a stable posture of opposing gripping or complementary support by both arms. Subsequently, the first and second robotic arms perform high-precision collaborative motion in a force-position hybrid control mode: both arms synchronously lift and adjust their posture, sharing the load weight to counteract the eccentric torque, smoothly removing the target material frame 8 vertically from the pallet 7 and transferring it to the designated transfer position. This strategy effectively solves the technical problem of traditional single-arm robots being unable to safely cut into narrow gaps or being prone to collisions, significantly improving the destabilization success rate and operational safety in complex stacking scenarios.

[0052] In one embodiment, the robotic arm execution mechanism 6 includes a first robotic arm and a second robotic arm; the palletizing strategy includes: identifying the current working scenario based on pose data; when the current working scenario is an initial empty pallet, determining the palletizing target pose based on the pose of the empty pallet; when the current working scenario is an existing pallet type, determining the palletizing target pose based on the pose of the top layer frame; when performing the palletizing operation of the same layer of frames, calculating the expected distance between the target frame 8 and the already placed frames; when the expected distance is greater than or equal to a safety threshold, controlling the first robotic arm and the second robotic arm to cooperate in placing the target frame 8 in the target pose and simultaneously unloading the target frame 8; when the expected distance is less than the safety threshold, controlling the first robotic arm and the second robotic arm to cooperate in transporting the target frame 8 to a preset position, controlling the gripper on the side close to the already placed adjacent frame 9 to release, and the other gripper to hold the target frame 8 alone; controlling the robotic arm on the holding side to move in the direction toward the already placed adjacent frame 9, pushing the target frame 8 until it contacts the already placed adjacent frame 9.

[0053] In one specific implementation, the robotic arm actuator 6 is configured as a dual-arm collaborative system comprising a first robotic arm and a second robotic arm, whose palletizing strategy features scene adaptation and precise gap adjustment. First, the system performs semantic recognition of the current work scene based on real-time pose data: if determined to be an initial empty pallet scene, the absolute palletizing target pose of the first layer of material frames is directly calculated based on the reference pose of the empty pallet; if determined to be an existing pallet scene, the pose information of the current top layer of material frames is dynamically extracted, and based on this, the relative palletizing target pose of the next layer or subsequent material frames in the same layer is calculated, ensuring the geometric consistency of the pallet stacking.

[0054] When performing the stacking operation of the same layer of material frames, the planning module pre-calculates the theoretical expected distance between the target material frame 8 and the adjacent material frame 9 after it is placed, and compares it with a preset safety threshold. Based on this, the execution mode is dynamically selected: when the expected distance is greater than or equal to the safety threshold, it indicates that the working space is sufficient and there is no risk of collision. At this time, the system controls the first and second robotic arms to maintain a dual-arm coordinated gripping state, accurately transporting the target material frame 8 above the target position. Then, the two grippers simultaneously perform the unloading action, smoothly placing the material frame in the predetermined position, completing the efficient standard stacking process.

[0055] When the expected gap is less than the safety threshold, it indicates that the target frame 8 needs to be embedded into the narrow gap with extremely high precision, and direct placement is prone to interference. At this time, the system initiates a "push-in" precision assembly strategy: First, the first and second robotic arms work together to transport the target frame 8 to a preset transition position near the target pose; then, the gripper of the robotic arm on the side closer to the already placed adjacent frame 9 is released, releasing the constraint on that side, while the gripper of the other robotic arm holds the target frame 8 alone, making it in a semi-free suspension state; finally, the control system instructs the robotic arm on the holding side to perform a slight translational movement in the direction towards the already placed adjacent frame 9, using single-arm thrust to push the target frame 8 into the narrow gap until the side of the target frame 8 is in close contact with the already placed adjacent frame 9 or reaches the set zero-gap fit state. This strategy, through the action sequence of "first positioning, then single-sided release, then push-in", effectively solves the placement failure problem caused by accumulated errors in high-density palletizing scenarios, and achieves tight stacking with millimeter-level precision.

[0056] In one implementation, the anomaly handling strategy includes: when a first abnormal state is triggered, controlling the robotic arm actuator 6 to stop its current motion trajectory and maintain its current position for a preset time; controlling the robotic arm actuator 6 to retreat a preset safe distance along the direction of the sudden change in operating force; after the retreat is completed, replanning the path for grabbing or placing the target material frame 8 and performing a compensation operation; if the number of consecutive compensation operations exceeds a preset retry threshold, triggering an emergency stop alarm; when a second abnormal state is triggered, controlling the robotic arm actuator 6 to pause its operation and move to a preset safe waiting posture; if the target material frame 8 is relocked within a preset time window, updating the motion plan based on the newly acquired posture data and resuming the operation; if the target material frame 8 is not locked within the time limit, triggering a manual intervention alarm; when it is detected in real time that both the first and second abnormal states have disappeared, controlling the robotic arm actuator 6 to continue performing subsequent actions along the motion trajectory.

[0057] First, when the system determines that the first abnormal state has been triggered, namely a collision or slippage caused by a sudden change in force feedback, the mechanical protection process is immediately initiated: the control system interrupts the current motion trajectory command with the highest priority, drives the robotic arm actuator 6 to brake urgently within milliseconds, and locks the current position within a preset holding time to eliminate inertial impact and stabilize the load posture. Subsequently, based on the direction of the sudden change vector recorded by the force sensor, the system controls the robotic arm to retract a preset safe distance in the opposite direction to quickly resolve potential rigid interference or jamming. After completing the retraction action, the path planning module recalculates the obstacle avoidance path for grasping or placing the target material frame 8 based on the currently updated environmental constraints and performs a compensation operation attempt. If the system fails to complete the task after continuously performing compensation operations exceeding the preset retry threshold, it is determined to be an unrecoverable hard fault, immediately triggering an emergency stop alarm and notifying the host computer to intervene.

[0058] Secondly, when the system determines that a second abnormal state has been triggered, namely the loss of visual tracking features, the vision recovery process is initiated: the control system immediately suspends the current work sequence and directs the robotic arm actuator 6 to smoothly move to a preset safe waiting position. This safe waiting position is usually located in an open area in the center of the workspace without collision risk, to avoid erroneous actions in blind operation. The system then enters a preset time window for feature re-lock monitoring: if the vision module successfully recaptures and locks the feature points of the target material frame 8 during this period, the system will dynamically update the motion planning parameters based on the newly acquired high-precision pose data, seamlessly resuming subsequent operations; if the target is not re-locked after the time window, it is determined that the vision guidance has failed, triggering a manual intervention alarm, prompting the operator to check for obstructions or recalibrate.

[0059] Finally, the system has an automatic recovery function for abnormal states: during the abnormal handling process, if the first abnormal state and the second abnormal state are both detected to have disappeared in real time, the system issues an instruction to make the robotic arm actuator 6 continue to perform subsequent actions along the original motion trajectory from the breakpoint or the optimized path node, thereby realizing the automated and continuous recovery of the work process.

[0060] In one implementation, the motion trajectory is corrected in real time based on the second environmental point cloud data to obtain an obstacle avoidance path, including: constructing a local cost map containing dynamic obstacles in real time based on the second environmental point cloud data, the dynamic obstacles including moving workers or target objects whose positions change; mapping the motion trajectory to the local cost map for collision detection; if a conflict is detected between the motion trajectory and the dynamic obstacle, the joint angle sequence of the robotic arm actuator 6 is replanned online in the joint space to generate an obstacle avoidance path.

[0061] In one specific implementation, a highly dynamic local perception and replanning closed-loop mechanism is constructed to generate obstacle avoidance paths by real-time correction of motion trajectories based on second environmental point cloud data. First, the system utilizes second environmental point cloud data acquired in real-time by a depth vision sensor and dynamically constructs a local cost map containing both static environment and dynamic obstacles through voxel filtering and temporal difference algorithms. In this map, the system specifically extracts features and updates probabilistic grids for dynamic obstacles such as moving workers, shuttling AGVs, or target objects whose positions change abruptly, assigning them a time-varying expansion radius and high cost value to reflect their movement trends and potential risk areas.

[0062] Subsequently, the planning module discretizes the current preset motion trajectory of the robotic arm actuator 6 into a series of spatiotemporal nodes and maps them to the aforementioned real-time updated local cost map for multi-dimensional collision detection. The system not only detects interference at geometric positions but also combines the robotic arm's motion speed with the predicted trajectory of dynamic obstacles to assess the probability of collisions within future time windows.

[0063] Once a conflict is detected between the preset trajectory and a dynamic obstacle—that is, when the collision probability exceeds the safety threshold or the cost exceeds the allowable range—the system immediately triggers an online replanning mechanism. This mechanism abandons the drawback of the long processing time of traditional global replanning, instead directly optimizing and adjusting the joint angle sequence of the robotic arm within the joint space. By employing a sampling-based or optimization-based local search algorithm, the system quickly calculates a new set of joint angle sequences that can bypass the high-cost areas of dynamic obstacles while keeping the constraints of the starting and target points unchanged. Finally, the system smoothly converts the generated new sequence into continuous obstacle avoidance path instructions, driving the robotic arm actuator 6 to deviate from the original trajectory in real time. This allows for flexible bypassing of dynamic interference while ensuring a safe distance, and automatically returns to or smoothly connects to the original work process after the obstacle is cleared, thereby achieving high safety and high continuity of operation in complex human-machine collaborative environments.

[0064] In one embodiment, the operating force of the robotic arm actuator 6 relative to the target object is dynamically adjusted based on force feedback data, including: during the grasping phase, controlling the robotic arm actuator 6 to move to a preset coarse positioning pose based on visual servoing; then switching to a constant force admittance control mode, calculating the pose correction amount and adjusting the end-effector pose of the robotic arm actuator 6 based on the contact force deviation collected in real time by the force feedback unit 4, until a fine positioning pose is reached and the grasping of the target object is completed; during the placement phase, controlling the robotic arm actuator 6 to move to a preset coarse positioning pose based on visual servoing; then switching to a constant force admittance control mode, calculating the pose correction amount and adjusting the end-effector pose of the robotic arm actuator 6 based on the contact force deviation collected in real time by the force feedback unit 4, so that the target object approaches the target placement surface with a constant contact force, until the placement of the target object is completed.

[0065] In one specific implementation, the operating force of the robotic arm actuator 6 relative to the target object is dynamically adjusted based on force feedback data. A phased adaptive control strategy combining visual guidance and force compliance is adopted to address the uncertainty of the target object's pose and the rigid impact of the contact surface.

[0066] During the grasping phase, the system first activates the visual servo control loop, driving the robotic arm actuator 6 to rapidly move to a preset coarse positioning pose based on the visually recognized characteristics of the target object. This pose is typically set in a safe approach area above or to the side of the target object's grasping point. At this time, the robotic arm is in position control mode, primarily addressing large-scale spatial positioning issues. Once the robotic arm enters the preset force sensing range or detects an initial contact signal, the control system immediately and seamlessly switches to Constant Force Admittance Control mode. In this mode, the system collects six-dimensional contact force data output by the force feedback unit 4 in real time and compares it with the preset ideal grasping contact force to calculate the real-time contact force deviation. Based on the admittance control algorithm, the system maps the aforementioned force deviation to the pose correction amount of the end effector, dynamically adjusting the robotic arm's trajectory to approximate the actual surface normal and geometry of the target object. This process continues until the robotic arm's end effector reaches the fine positioning pose and stably establishes a reliable grasping force, ultimately completing the flexible grasping of the target object and effectively avoiding hard collisions or grasping failures caused by visual errors or object deformation.

[0067] During the placement phase, the system also employs high-precision force-sensing interaction logic. As the robotic arm carrying the target object moves above the target placement surface and begins to descend, the system continuously monitors the output signal of the force feedback unit 4. Once the contact force signal reaches the preset initial contact threshold, indicating that the end effector or load has slightly touched the placement surface, the control system immediately locks and maintains a constant force admittance control mode. In this mode, the robotic arm no longer relies solely on position commands to forcefully press down, but instead uses a set constant contact force as the control target, controlling the end effector to smoothly approach and ultimately conform to the target placement surface. As the placement action progresses, the robotic arm fine-tunes its Z-axis height and attitude angle based on real-time force feedback, ensuring that the target object lands smoothly and without impact, until the load is completely released and the contact force returns to the baseline state, thus completing a high-precision, non-destructive placement operation. This strategy, through force-position hybrid control, significantly improves the robotic arm's adaptability to handling fragile, deformable, or high-precision mating objects in unstructured environments.

[0068] like Figure 3As shown, according to a second embodiment of the present invention, a humanoid robot system 1 is provided for performing the method of any of the above embodiments. The system includes: an environment perception unit 2 for scanning the work area and generating first environmental point cloud data representing a static map; a vision perception unit 3 for acquiring pose data of a target object; a force feedback unit 4 for detecting force feedback data of a robotic arm actuator 6 during contact with the target object; the force feedback unit 4 is integrated at the end of the robotic arm actuator 6; and a central control unit 5 for determining whether an abnormal state is triggered based on the pose data and the force feedback data; when no abnormal state is triggered, identifying the arrangement type of the target material frame in the work area based on the pose data; matching a destacking strategy or palletizing strategy corresponding to the material frame arrangement type from a preset work strategy library as the current execution strategy; when an abnormal state is triggered, selecting an abnormal handling strategy as the current execution strategy; and combining the pose data with the current execution strategy. The system uses the spatial coordinates and attitude angle of the target material box in the image, along with the first environmental point cloud data, to calculate the end effector path and joint motion sequence of the robotic arm, generating the motion trajectory of the robotic arm actuator 6. During the pose motion of the robotic arm actuator 6, the system controls the environmental perception unit 2 to scan the work area in real time and generate second environmental point cloud data representing the distribution of dynamic obstacles. The motion trajectory is corrected in real time based on the second environmental point cloud data to obtain an obstacle avoidance path. The system controls the chassis 11 of the humanoid robot based on a static map, enabling the humanoid robot to move to the work area. A waist unit 10 is connected between the chassis 11 and the robotic arm actuator 6. The waist unit 10 is used to drive the lifting and pitch angles of the robotic arm actuator. The robotic arm actuator 6 continues to perform pose motion along the real-time obstacle avoidance path. The system dynamically adjusts the operating force of the robotic arm actuator 6 relative to the target object based on force feedback data, and calculates the pose correction amount and adjusts the end effector pose of the robotic arm actuator 6 based on force feedback data.

[0069] The environmental perception unit 2 is used to perform a wide-area scan of the work area and generate first environmental point cloud data representing a static map. In specific implementations, this unit may include a LiDAR (Light Detection and Ranging) or depth camera module installed on the head or torso of the humanoid robot. For example, a multi-line rotating LiDAR or solid-state LiDAR can be used, which can perform a 360-degree or large field-of-view three-dimensional scan of the surrounding environment with the robot body as the center, constructing a high-precision point cloud map including static facilities such as walls, fixed shelves, and stationary pallets, providing static environmental constraints for global path planning.

[0070] The visual perception unit 3 is used to collect and acquire the pose data of target objects, such as the material frame and pallet 7. In specific implementations, this unit may include a color depth (Red Green Blue-D) camera or a binocular stereo vision sensor integrated into the robot's wrist, elbow, or head. For example, a structured light depth camera or an active stereo vision camera can be used, which can capture the texture image and depth information of the target object at close range, calculate the three-dimensional coordinates, pose angles, and size parameters of the target object through image processing algorithms, and transmit this pose data to the central control unit 5 in real time as a reference input for grasping and palletizing.

[0071] The force feedback unit 4 is used to detect force feedback data of the robotic arm actuator 6 during contact with a target object, and this unit is directly integrated into the end effector of the robotic arm actuator 6. In a specific implementation, this unit can be a six-dimensional force / torque sensor, which is installed between the flange and the end effector gripper or suction cup at the end of the robotic arm. This sensor can sense the force components of the end effector in the X, Y, and Z directions and the torque components around these three axes in real time, converting the mechanical signals generated by physical contact into electrical signals for constant force admittance control, collision detection, and force adjustment in precision assembly.

[0072] The central control unit 5, as the computational core of the system, generates the initial motion trajectory of the robotic arm actuator 6 based on the received pose data, the first environmental point cloud data, and the preset operation strategy library. During the pose motion of the robotic arm, it simultaneously controls the environmental perception unit 2 to continuously scan the work area in real time, generating second environmental point cloud data representing dynamic obstacles, such as the distribution of moving personnel or other mobile devices. Then, based on the second environmental point cloud data, it performs real-time correction and replanning of the initial motion trajectory to calculate a collision-free obstacle avoidance path. In specific implementations, this unit may include a main control computer or embedded industrial control computer equipped with a high-performance processor, which internally runs motion planning algorithms, Simultaneous Localization and Mapping (SLAM) modules, and anomaly handling logic modules, responsible for coordinating data fusion and decision-making from various sensors.

[0073] The robotic arm actuator 6 performs pose movements along a real-time obstacle avoidance path generated by the central control unit 5, and dynamically adjusts the operational force relative to the target object based on data provided by the force feedback unit 4. In a specific implementation, this mechanism may include a humanoid dual-arm structure, with each arm consisting of multiple rotary joints equipped with high-precision servo motors, and the ends connected to electric grippers or flexible grippers adapted to the size of the material frame. During task execution, the robotic arm moves according to the planned path, and at the moment of contact, based on feedback data from the six-dimensional force sensor, adjusts the joint output torque to achieve compliant gripping, push-in stacking, or force-controlled retraction after collision, ensuring the smoothness and safety of the operation process.

[0074] In some examples, the robot system consists of an omnidirectional moving chassis 11, a height-adjustable and pitch-capable waist unit 10, dual seven-DOF robotic arms, and a head gimbal. The chassis 11 is equipped with a 3D LiDAR for real-time positioning, navigation, and dynamic obstacle avoidance in factory environments. The waist and arms work together to cover multi-level palletizing and depalletizing operations. At the perception level, a binocular vision configuration is used: a high-precision RGB-D camera on the head, calibrated by hand and eye, is dedicated to 6D pose calculation of the material frame, while a wide-angle 3D camera on the chest is responsible for constructing a dense point cloud around the robot, providing real-time obstacle avoidance data for motion planning. At the end effector, a six-dimensional force sensor is embedded in a flange interface to form a force feedback closed loop, supporting admittance control strategies. It is also equipped with a universal quick-change device, which can flexibly switch between specialized grippers such as clamps, hooks, or suction devices according to material characteristics. All perception and control algorithms run in a multi-threaded real-time central controller embedded in the torso, achieving end-to-end coordination from task decision-making to underlying drive.

[0075] During the transport phase, based on the robot's kinematic model and real-time updated point cloud data, the robot's chassis 11, waist, and robotic arm actuator 6 are subjected to full-body coordinated motion planning to generate a collision-free movement trajectory. The robot is then controlled to move the grasped target object to the target area along the movement trajectory.

[0076] A third embodiment of the present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method of any of the above embodiments.

[0077] Figure 4 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, electronic device 900 may be provided as a server. (Refer to...) Figure 4The electronic device 900 includes a processing component 922, which further includes one or more processors, and memory resources represented by memory 932 for storing instructions, such as application programs, that can be executed by the processing component 922. The application programs stored in memory 932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 922 is configured to execute instructions to perform the methods described above.

[0078] Electronic device 900 may also include a power supply component 926 configured to perform power management of electronic device 900, a wired or wireless network interface 950 configured to connect electronic device 900 to a network, and an input / output (I / O) interface 958. Electronic device 900 may operate on an operating system stored in memory 932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0079] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 932 including instructions, which can be executed by a processing component 922 of an electronic device 900 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0080] A fourth embodiment of the present invention provides a readable storage medium storing a program, which, when executed, implements the method of any of the above embodiments.

[0081] The fifth embodiment of the present invention provides a computer program product, including a computer program, which, when executed, implements the method of any of the above embodiments.

[0082] In this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0083] The above description of the embodiments is intended to enable those skilled in the art to understand and apply the present invention. It will be apparent to those skilled in the art that various modifications can be made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, the present invention is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope and spirit of the invention are within the scope of the present invention.

Claims

1. A control method for a humanoid robot, applied to a humanoid robot system comprising a visual perception unit, an environmental perception unit, a robotic arm actuator, and a force feedback unit, characterized in that, The control method includes: The environmental perception unit is controlled to scan the work area and generate first environmental point cloud data representing a static map. The visual perception unit is controlled to acquire the pose data of the target object; The control unit integrated at the end of the robotic arm actuator detects the force feedback data of the robotic arm actuator during contact with the target object. Based on the pose data and the force feedback data, determine whether an abnormal state is triggered; when no abnormal state is triggered, identify the arrangement type of the target material frame in the work area based on the pose data; match the destacking strategy or palletizing strategy corresponding to the material frame arrangement type from the preset work strategy library as the current execution strategy; when an abnormal state is triggered, select the abnormal handling strategy as the current execution strategy. Based on the current execution strategy, combined with the spatial coordinates and attitude angle of the target frame in the pose data, and the first environmental point cloud data, the end effector path and joint motion sequence of the robotic arm are calculated to generate the motion trajectory of the robotic arm actuator. During the process of driving the robotic arm actuator to perform pose movement, the environmental perception unit is controlled to scan the working area in real time and generate second environmental point cloud data representing the distribution of dynamic obstacles; the motion trajectory is corrected in real time based on the second environmental point cloud data to obtain an obstacle avoidance path; and the robotic arm actuator is controlled to continue performing pose movement along the real-time obstacle avoidance path. The robotic arm actuator is dynamically adjusted relative to the target object based on the force feedback data.

2. The control method according to claim 1, characterized in that, The target object includes a target frame and a target placement position, and the pose data includes the target grasping pose and the target placement pose. Controlling the visual perception unit to acquire the pose data of the target object includes: After the humanoid robot moves to the target work area, the visual perception unit is controlled to scan the target object and obtain raw point cloud data including the target material frame and the target placement position. The original point cloud data is segmented to separate the first point cloud cluster corresponding to the target material frame and the second point cloud cluster corresponding to the target placement position; Based on the first point cloud cluster, the grasping surface and grasping points of the target material frame are identified, and the target grasping pose of the target material frame in the robot base coordinate system is calculated. Based on the second point cloud cluster, the target placement pose in the robot base coordinate system is calculated.

3. The control method according to claim 1, characterized in that, During the process of driving the robotic arm actuator to perform pose movements, the environmental perception unit is controlled to scan the work area in real time and generate second environmental point cloud data representing the distribution of dynamic obstacles, including: The humanoid robot is driven to move to the work area based on the static map. Upon reaching the work area, the environmental perception unit is controlled to perform a three-dimensional scan of the current work area to obtain first environmental point cloud data including surrounding obstacles. The first environmental point cloud data is compared and analyzed with the static map. Background points and dynamic interference points that overlap with the static map are removed, and obstacle features that have changed or been added relative to the static map are extracted to generate the second environmental point cloud data.

4. The control method according to claim 1, characterized in that, The step of determining whether an abnormal state is triggered based on the pose data and the force feedback data includes: The force feedback data is detected in real time. When the operating force of the robotic arm actuator relative to the target object suddenly exceeds the preset force threshold, it is determined that a collision or grasping slippage has occurred, triggering the first abnormal state. The pose data is monitored in real time, and a second abnormal state is triggered when the tracking features of the target object are detected to be lost.

5. The control method according to claim 1, characterized in that, The robotic arm actuator includes a first robotic arm and a second robotic arm; the destacking strategy includes: When the distance between the target material frame and the adjacent material frame is less than or equal to the safety threshold, the first robotic arm is controlled to grasp the first side of the target material frame and pull the target material frame to a preset separation distance in a direction away from the adjacent material frame. The second robotic arm is then controlled to grasp the second side of the target material frame. The first side and the second side are the opposite sides of the target material frame. When it is confirmed that the distance is greater than the safety threshold, the first robotic arm and the second robotic arm are controlled to work together to remove the target material frame from the pallet.

6. The control method according to claim 1, characterized in that, The robotic arm actuator includes a first robotic arm and a second robotic arm; the palletizing strategy includes: The current work scenario is identified based on the pose data; when the current work scenario is an initial empty pallet, the palletizing target pose is determined based on the pose of the empty pallet; when the current work scenario is an existing pallet type, the palletizing target pose is determined based on the pose of the top layer frame. When performing the stacking operation of the same layer of material frames, calculate the expected distance between the target material frame and the already placed material frames; When the expected spacing is greater than or equal to the safety threshold, the first robotic arm and the second robotic arm are controlled to work together to place the target material frame in the target pose and simultaneously unload the target material frame. When the expected distance is less than the safety threshold, the first robotic arm and the second robotic arm are controlled to work together to transport the target material frame to the preset position. The gripper on the side near the adjacent material frame is controlled to release, and the other gripper holds the target material frame alone. The robotic arm on the holding side is controlled to move in the direction toward the adjacent material frame and push the target material frame until it contacts the adjacent material frame.

7. The control method according to claim 4, characterized in that, The exception handling strategy includes: When the first abnormal state is triggered, the robotic arm actuator is controlled to stop its current movement trajectory and maintain its current position for a preset time; the robotic arm actuator is controlled to retreat a preset safe distance along the direction of the sudden change in the operating force; after the retreat is completed, the path for grabbing or placing the target material box is replanned and a compensation operation is performed; if the number of consecutive compensation operations exceeds the preset retry threshold, an emergency stop alarm is triggered. When the second abnormal state is triggered, the robotic arm actuator is controlled to suspend operation and move to a preset safe waiting position; if the target material box is relocked within the preset time window, the motion plan is updated based on the newly acquired position data and the operation is resumed; if the target material box is not locked within the time limit, a manual intervention alarm is triggered. When it is detected in real time that both the first abnormal state and the second abnormal state have disappeared, the robotic arm actuator is controlled to continue to perform subsequent actions along the motion trajectory.

8. The control method according to claim 1, characterized in that, The motion trajectory is corrected in real time based on the second environmental point cloud data to obtain an obstacle avoidance path, including: Based on the second environmental point cloud data, a local cost map containing dynamic obstacles is constructed in real time, including moving workers or target objects whose positions change. The motion trajectory is mapped onto the local cost map for collision detection; If a conflict is detected between the motion trajectory and the dynamic obstacle, the joint angle sequence of the robotic arm actuator is replanned online in the joint space to generate the obstacle avoidance path.

9. The control method according to claim 1, characterized in that, Dynamically adjusting the operating force of the robotic arm actuator relative to the target object based on the force feedback data includes: During the grasping phase, the robotic arm actuator is controlled to move to a preset coarse positioning pose based on visual servo motion; then it switches to constant force admittance control mode, calculates the pose correction amount and adjusts the end pose of the robotic arm actuator based on the contact force deviation collected in real time by the force feedback unit, until the fine positioning pose is reached and the grasping of the target object is completed. During the placement phase, the robotic arm actuator is controlled to move to a preset coarse positioning pose based on visual servo motion; then it switches to constant force admittance control mode, calculates the pose correction amount and adjusts the end pose of the robotic arm actuator based on the contact force deviation collected in real time by the force feedback unit, so that the target object approaches the target placement surface with a constant contact force until the placement of the target object is completed.

10. A humanoid robot system for performing the method of any one of claims 1 to 9, characterized in that, The system includes: An environmental perception unit is used to scan the work area and generate first environmental point cloud data representing a static map. The visual perception unit is used to acquire the pose data of the target object. A force feedback unit is used to detect force feedback data of the robotic arm actuator during contact with a target object; the force feedback unit is integrated into the end effector of the robotic arm actuator. The central control unit is used to determine whether an abnormal state is triggered based on the pose data and the force feedback data; when no abnormal state is triggered, it identifies the arrangement type of the target material frame in the work area based on the pose data; it matches the destacking or palletizing strategy corresponding to the material frame arrangement type from a preset work strategy library as the current execution strategy; when an abnormal state is triggered, it selects an abnormal handling strategy as the current execution strategy; based on the current execution strategy, combined with the spatial coordinates and posture angle of the target material frame in the pose data, and the first environmental point cloud data, it calculates the end effector path and joint motion sequence of the robotic arm, and generates the motion trajectory of the robotic arm actuator; during the process of driving the robotic arm actuator to perform pose motion, it controls the environmental perception unit to scan the work area in real time and generate second environmental point cloud data representing the distribution of dynamic obstacles; it corrects the motion trajectory in real time based on the second environmental point cloud data to obtain an obstacle avoidance path; and it controls the chassis of the humanoid robot according to the static map to make the humanoid robot move to the work area. A waist unit is connected between the chassis and the robotic arm actuator; the waist unit is used to drive the lifting and pitching angles of the robotic arm actuator. The robotic arm actuator is used to continue performing pose movement along the real-time obstacle avoidance path; dynamically adjust the operating force of the robotic arm actuator relative to the target object according to the force feedback data; and calculate the pose correction amount according to the force feedback data and adjust the end pose of the robotic arm actuator.

11. An electronic device, characterized in that, The system includes a memory and a processor, wherein the memory stores a computer program executable by the processor; and the processor executes the computer program in the memory to implement the method as described in any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the executable computer program in the storage medium is executed by a processor, it can implement the method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 9.

Images