Robot dynamic obstacle avoidance method, device and equipment

Abstract

The application provides a robot dynamic obstacle avoidance method, device and equipment, comprising: acquiring position information of key points of double arms of an operator, and constructing a double arm geometric model based on the key points; wherein the key points comprise a first key point and a second key point for respectively representing a left hand and a right hand of the operator; determining a risk region state according to relative positions of the first key point and the second key point; determining a target repulsive force according to a distance between the left hand and the right hand and a relative position relationship between a robot and the risk region; performing smooth updating on the target repulsive force according to an actual repulsive force of a previous period to obtain an actual repulsive force of a current period; performing vector composition on the actual repulsive force of the current period and an attractive force generated by a target point to obtain a resultant force suffered by the robot; determining a motion direction of the robot according to the resultant force, and controlling the robot to perform obstacle avoidance motion.

Description

Technical Field

[0001] This invention relates to the field of human-machine collaboration technology, and in particular to a method, apparatus and equipment for dynamic obstacle avoidance of robots. Background Technology

[0002] With the advancement of intelligent manufacturing, although industrial automated production lines have widely adopted robots to replace manual operations, full automation is still not achievable. For example, tasks requiring extremely high precision, such as assembly and disassembly, still rely on the flexibility and fine motor skills of humans. However, relying solely on manual operation is extremely inefficient and cannot meet the demands of large-scale production. Therefore, the Human-Robot Collaboration (HRC) model has emerged, providing an ideal paradigm for achieving both high efficiency and flexibility in production by combining the high repeatability of robots with the cognitive flexibility of humans. At the same time, it also brings more severe challenges, placing higher demands on the safety of collaborative operations.

[0003] It is worth noting that most existing research on human-robot collaboration focuses primarily on scenarios where a human operator works with a robot using only one arm, failing to fully consider the complexity and unique challenges of two-arm collaborative operation. For example, in tasks such as disassembling electronic products, assembling and testing aircraft, and sorting workpieces, human operators need to use both arms for collaborative work. Forcing single-arm operation would not only reduce system efficiency but could also lead to malfunctions in collaborative movements, increasing the risk of physical collisions and misoperations during the operation.

[0004] In scenarios where an operator's arms work together simultaneously, the relative positions of the hands continuously change during operation, causing corresponding changes in the robot's safety boundaries and passable space. Existing robot obstacle avoidance methods typically rely on the currently detected obstacle position or geometric boundaries. When the hands are close together, gaps insufficient for safe passage are easily misjudged as passable areas; conversely, when the hands are far apart, the obstacle avoidance range may be too large, excluding usable space. Furthermore, during the dynamic changes in the arm positions and their relationship, the obstacle boundaries upon which the robot operates and the resulting repulsive forces may change significantly, potentially causing abrupt changes in the robot's movement direction or obstacle avoidance control parameters, affecting the continuity and stability of the dynamic obstacle avoidance process. Therefore, existing technologies struggle to simultaneously address both the adaptability of the safe space under changing arm relative positions and the stability of robot obstacle avoidance control. Summary of the Invention

[0005] This invention provides a method, apparatus, and device for dynamic obstacle avoidance of a robot, which addresses the shortcomings of existing technologies where, when an operator performs dual-arm operations in collaboration with a robot, it is difficult to simultaneously ensure the safety space adaptability under changes in the relative positions of the two arms and the stability of the robot's obstacle avoidance control.

[0006] This invention provides a dynamic obstacle avoidance method for a robot, comprising: acquiring the positional information of key points of an operator's two arms, and constructing a geometric model of the two arms based on the key points; wherein the key points include a first key point and a second key point respectively representing the operator's left and right hands; determining a risk area state based on the relative positions of the first and second key points; wherein, in the first risk area state, the risk area is a joint risk area surrounding the left and right hands and the gap between them, and in the second risk area state, the risk area includes two independent risk areas surrounding the left and right arms respectively; determining a target repulsive force based on the distance between the left and right hands and the relative positional relationship between the robot and the risk areas; smoothly updating the target repulsive force based on the actual repulsive force of the previous cycle to obtain the actual repulsive force of the current cycle; vector-combining the actual repulsive force of the current cycle with the gravitational force generated by the target point to obtain the resultant force on the robot; determining the robot's motion direction based on the resultant force, and controlling the robot to perform obstacle avoidance motion.

[0007] According to the robot dynamic obstacle avoidance method provided by the present invention, the step of determining the risk area state based on the relative positions of the first key point and the second key point includes: determining the risk area state as the first risk area state when the distance between the left hand and the right hand is less than or equal to a preset distance threshold; and determining the risk area state as the second risk area state when the distance between the left hand and the right hand is greater than the preset distance threshold.

[0008] The robot dynamic obstacle avoidance method provided by the present invention includes: when the risk area state is determined to be the first risk area state, the risk area is determined to be a spherical area; the center of the spherical area is the midpoint of the line connecting the first key point and the second key point.

[0009] According to the robot dynamic obstacle avoidance method provided by the present invention, the key points further include a third key point and a fourth key point for characterizing the operator's left wrist and right wrist, respectively, and a fifth key point and a sixth key point for characterizing the end of the operator's left forearm and right forearm; the method further includes: when the risk area state is determined to be the second risk area state, determining a first ellipsoidal risk area surrounding the left arm based on the first key point, the third key point and the fifth key point, and determining a second ellipsoidal risk area surrounding the right arm based on the second key point, the fourth key point and the sixth key point.

[0010] According to the robot dynamic obstacle avoidance method provided by the present invention, determining the target repulsion force based on the distance between the left and right hands and the relative positional relationship between the robot and the risk area includes: determining a binary state factor based on the relationship between the distance between the left and right hands and the preset distance threshold; determining the currently effective risk area based on the value of the binary state factor; and determining the target repulsion force based on the relative positional relationship between the robot and the currently effective risk area.

[0011] According to the robot dynamic obstacle avoidance method provided by the present invention, determining the binary state factor based on the relationship between the distance between the left and right hands and the preset distance threshold includes: determining the binary state factor as a first state value when the distance is less than or equal to the preset distance threshold; determining the value of the binary state factor of the previous cycle as the value of the binary state factor of the current cycle when the distance is greater than the preset distance threshold and less than the sum of the preset distance threshold and the hysteresis bandwidth; and determining the binary state factor as a second state value when the distance is greater than or equal to the sum of the preset distance threshold and the hysteresis bandwidth.

[0012] According to the robot dynamic obstacle avoidance method provided by the present invention, determining the target repulsive force based on the relative positional relationship between the robot and the currently effective risk area includes: determining the shortest distance between the robot and the surface of the currently effective risk area and the repulsive force direction corresponding to the shortest distance; determining the repulsive force corresponding to the currently effective risk area based on the shortest distance and the repulsive force direction; and determining the repulsive force corresponding to the current risk area state as the target repulsive force based on the value of the binary state factor.

[0013] According to the robot dynamic obstacle avoidance method provided by the present invention, the step of smoothly updating the target repulsion force based on the actual repulsion force of the previous cycle to obtain the actual repulsion force of the current cycle includes: weighting and summing the target repulsion force and the actual repulsion force of the previous cycle according to the repulsion force smoothing coefficient to obtain the actual repulsion force of the current cycle.

[0014] This invention also provides a robot dynamic obstacle avoidance device, comprising: a key point acquisition module, used to acquire the position information of key points of the operator's two arms, and construct a geometric model of the two arms based on the key points; wherein the key points include a first key point and a second key point respectively representing the operator's left and right hands; a risk state determination module, used to determine the risk area state according to the relative position of the first key point and the second key point; wherein, in the first risk area state, the risk area is a joint risk area surrounding the left and right hands and the gap between them, and in the second risk area state, the risk area includes two independent risk areas respectively surrounding the left and right arms; a target repulsion force determination module, used to determine the target repulsion force according to the distance between the left and right hands and the relative positional relationship between the robot and the risk area; a resultant force calculation module, used to smoothly update the target repulsion force according to the actual repulsion force of the previous cycle to obtain the actual repulsion force of the current cycle; and to vector synthesize the actual repulsion force of the current cycle with the attraction force generated by the target point to obtain the resultant force on the robot; and an obstacle avoidance module, used to determine the robot's motion direction according to the resultant force and control the robot to perform obstacle avoidance motion.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the robot dynamic obstacle avoidance method as described above.

[0016] The robot dynamic obstacle avoidance method, apparatus, and device provided by this invention acquires the positional information of key points on the operator's arms and dynamically determines the state of the risk area between a joint risk area and two independent risk areas based on the relative positions of the left and right hands. This allows the risk area to change with the spatial relationship of the arms, incorporating the potential danger space between the hands into the obstacle avoidance range when the hands are close together, and avoiding excessive restriction on the available space between the hands when they are far apart. Furthermore, the target repulsive force is determined based on the distance between the hands and the relative position of the robot to the current risk area, enabling the obstacle avoidance action on the robot to adapt to the dynamic changes of the arms and the risk area. Simultaneously, the target repulsive force is smoothly updated based on the actual repulsive force of the previous cycle, and the obtained actual repulsive force is synthesized with the attraction generated by the target point. This reduces the abrupt changes in repulsive force caused by the movement of the arms and the changes in the state of the risk area, allowing the robot to maintain the continuity and stability of the movement process while dynamically avoiding obstacles on the operator's arms. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 A flowchart of a robot's dynamic obstacle avoidance method is shown. Figure 2 A diagram illustrating a human-computer collaboration scenario is shown. Figure 3 A schematic diagram illustrating a geometric modeling of a dual-arm system according to an example of the present invention is shown, wherein, Figure 3 (a) shows a schematic diagram of the key points of the arms. Figure 3 (b) shows a schematic diagram of constructing a geometric model of the two arms based on the key points of the two arms; Figure 4 A schematic diagram of a dynamic risk area according to an example of the present invention is shown, wherein, Figure 4 (a) shows a schematic diagram of the joint risk area formed when the hands are close together. Figure 4 (b) shows a schematic diagram of two separate risk zones formed when the hands are far apart; Figure 5 A schematic diagram illustrating the risk area coverage of an example of the present invention is shown, wherein, Figure 5 (a) shows a schematic diagram of the joint risk area. Figure 5 (b) shows a schematic diagram of the independent risk areas; Figure 6 A schematic diagram of a repulsive field according to an example of the present invention is shown, wherein, Figure 6 (a) shows a schematic diagram of the repulsive field formed based on the double-arm geometric model. Figure 6 (b) shows a schematic diagram of the repulsive field formed by combining dynamic risk regions; Figure 7 A schematic diagram of a collaborative platform according to an example of the present invention is shown; Figure 8 An example of the robot's dynamic obstacle avoidance verification diagram of the present invention is shown, wherein, Figure 8 (a) to Figure 8 (f) shows the robot's dynamic obstacle avoidance process at the stages of cooperation start, two arms approaching each other, avoiding the right arm, avoiding both arms, avoiding the left arm, and cooperation end. Figure 9 The diagram shows the curves illustrating the changes in joint angles and obstacle avoidance distance of a robot according to an example of the present invention. Figure 9 (a) shows a graph illustrating the changes in the angles of the robot's joints. Figure 9(b) shows a graph illustrating the change in the robot's obstacle avoidance distance; Figure 10 A comparison chart of the dynamic risk area usage in an example of the present invention is shown, wherein, Figure 10 (a) and Figure 10 (b) The obstacle avoidance behavior of the robot is shown when both arms are close together and when both arms are far apart, respectively, using a dynamic risk area. Figure 10 (c) shows Figure 10 (a) is a magnified view of a portion of the image. Figure 10 (a )and Figure 10 (b The diagrams show the robot's obstacle avoidance behavior when both arms are close together and when both arms are far apart, without the use of dynamic risk zones. Figure 10 (c ) shows Figure 10 (a A magnified view of a portion of the image; Figure 11 A structural block diagram of the robot's dynamic obstacle avoidance device is shown. Figure 12 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0020] In scenarios involving human arm collaboration, existing technologies often simply treat the arms as independent dynamic obstacles. However, when the arms move towards each other, the robot may mistake the narrow area between the hands for a passable one, which could lead to the risk of the robot coming into contact with the operator's hands.

[0021] In view of this, the present invention provides a method for dynamic obstacle avoidance of robots.

[0022] Figure 1 A flowchart of a robot's dynamic obstacle avoidance method is shown. Figure 2 A diagram illustrating a human-computer collaboration scenario is shown. Figure 3 The diagram illustrates the geometric modeling of the two arms. In the diagram, labels 1, 2, and 3 represent the center keypoint of the right hand, the keypoint of the right wrist, and the keypoint of the distal end of the right forearm, respectively. Labels 4, 5, and 6 represent the center keypoint of the left hand, the keypoint of the left wrist, and the keypoint of the distal end of the left forearm, respectively. The key point is the center of the hand. For the wrist, This is a key point at the end of the forearm; Figure 4 A schematic diagram of a dynamic risk area according to an example of the present invention is shown, wherein, The distance between the first keypoint and the second keypoint. This is the preset spacing threshold.

[0023] like Figure 1 As shown, the method includes operations S110~S150.

[0024] In operation S110, the position information of the key points of the operator's two arms is obtained, and a geometric model of the two arms is constructed based on the key points; wherein, the key points include the first key point and the second key point used to represent the operator's left hand and right hand, respectively.

[0025] According to embodiments of the present invention, the key points of the operator's arms can be obtained through open-source or commercial human pose estimation frameworks based on deep learning. For example, MediaPipe pose feature point detection and skeletal tracking technology can be used to track multiple joints of the operator's arms in real time and extract the three-dimensional spatial coordinate information of the key points. It should be understood that the technical solution of the pose feature point detection module is not limited to MediaPipe. Any component capable of achieving high-precision three-dimensional joint point tracking of the arms, including but not limited to OpenPose, YOLO-Pose, or a depth camera-based skeletal tracking SDK, falls within the equivalent substitution and protection scope of this invention.

[0026] According to an embodiment of the present invention, depth information of each key point can be obtained by a depth camera, and the two-dimensional coordinates can be mapped to three-dimensional coordinates in the robot base coordinate system by combining the camera intrinsic parameters. Furthermore, a dual-arm geometric model can be constructed based on the three-dimensional coordinates of the key points.

[0027] For example, the key points of the operator's arms may include the center key points of the left and right hands, namely the first key point and the second key point; the wrist key points of the left and right hands, namely the third key point and the fourth key point; the end key point of the left forearm, namely the fifth key point; and the end key point of the right forearm, namely the sixth key point.

[0028] In one example, the geometric model of the two arms can include a hand model and a forearm model. The hand model can be a sphere constructed with the center keypoint of the hand as the center. The forearm model can be a cylinder constructed with the wrist keypoint and the end keypoint of the forearm as the centers of the upper and lower surfaces, and the line connecting the two points as the central axis.

[0029] The sphere and cylinder together form an indivisible three-dimensional geometric model of two arms. This model distinguishes between the hand and arm parts while retaining the main shape features of the arm outline.

[0030] In this embodiment, the diameters of the sphere and cylinder are set to constant values ​​based on the arm circumference of a normal adult, while the arm length is dynamically determined by the real-time detection results of key points. This setting standardizes the differences in body size among different operators, avoids individual differences affecting subsequent obstacle avoidance performance, and thus improves the robustness of the method.

[0031] Alternatively, the geometric model can use an ellipsoid instead of a sphere to better fit the irregular shape of the hand. For example, the hand model can be a triaxial ellipsoid centered on the central key point of the hand, with its major axis extending along the palm direction.

[0032] Optionally, the diameter of the cylindrical model does not have to be a constant value, but can be dynamically adjusted based on the arm circumference estimate obtained from keypoint detection. For example, the arm diameter can be calculated in real time and mapped onto the cylindrical model by detecting the distance between the edge points of the arm contour.

[0033] In operation S120, the risk area status is determined based on the relative positions of the first key point and the second key point; wherein, in the first risk area status, the risk area is a joint risk area surrounding the left hand, the right hand and the gap between them, and in the second risk area status, the risk area includes two independent risk areas surrounding the left arm and the right arm respectively.

[0034] According to embodiments of the present invention, considering the complexity of dual-arm collaboration, which involves coordination and interaction between the arms (including moving closer or further apart), a phenomenon not present in single-arm scenarios, if a single-arm obstacle avoidance strategy is used, when the operator's arms are sufficiently close but still maintain a gap, the robot may misjudge this gap as a passable path and fail to perform obstacle avoidance. However, the actual width of this gap is insufficient for the robot to pass safely, leading to a physical collision. When the operator's arms are far apart, the robot may be unable to avoid the other arm in time after avoiding one arm, also resulting in a collision. Furthermore, when the hands are close together, the repulsive fields corresponding to the two independent risk areas will act simultaneously in the local space: the opposing repulsive forces between the hands may partially cancel each other out and form an incorrect passage area; the same-direction repulsive forces on the same side of the hands may repeatedly superimpose and lead to excessive detours; and detection noise or slight hand tremors may cause the resultant force direction to repeatedly change between the left and right sides, thereby causing path jitter.

[0035] To address the aforementioned issues, this embodiment proposes a dynamic risk area adjustment strategy. This strategy determines the hand-hand coordination based on the distance between the operator's hands and dynamically adjusts the merging or separation of risk areas and their geometry accordingly. This strategy does not simply scale a fixed envelope; instead, it alters the number of risk areas, the envelope object, and the boundary shape based on the hand-hand distance, ensuring that the safety boundary used for repulsion calculations aligns with the actual hand-hand coordination state.

[0036] For example, a preset distance threshold between the operator's left and right hands can be determined based on the size parameters of the robot's end effector and tools. When the distance between the first and second key points is less than or equal to this preset distance threshold, the risk area is considered to be in the first risk area state. The originally independent ellipsoidal risk areas can be merged and dynamically adjusted into a joint risk area surrounding the left and right hands and the gap between them. When the robot approaches this risk area, it performs obstacle avoidance, i.e., detours around the entire joint risk area. This allows the robot to calculate the effective distance and outward normal repulsion force only for a continuous closed boundary, geometrically closing gaps that are insufficient for safe passage, and preventing two independent repulsion fields from canceling each other out, overlapping, or alternately dominating in local space. This reduces erroneous passage, excessive detours, and path jitter, making the obstacle avoidance direction near the arms clearer and more stable.

[0037] When the distance between the first and second key points is greater than the preset distance threshold, the risk area state enters the second risk area state, indicating that the two arms have moved far enough away and can form independent obstacle boundaries. At this time, the joint risk area is closed and the two independent risk areas are activated. When the robot approaches the left or right arm, it performs obstacle avoidance respectively, while retaining the middle area between the two arms that meets the requirements for safe passage. This avoids unnecessary detours caused by an excessively large joint envelope, improves the utilization rate of shared space and the robot's passage efficiency.

[0038] According to an embodiment of the present invention, the geometric envelope shape of the merged joint risk area and the separated independent risk area can be adaptively adjusted to a geometric configuration such as a convex hull polyhedron, a capsule, or a combination of cylinders, based on the actual operator's body shape, clothing, or the contour of a specific assembly tool. These transformations and equivalent substitutions of geometric envelope shapes can also realize the dynamic risk area adjustment logic of the present invention.

[0039] When operating S130, the target repulsive force is determined based on the distance between the left and right hands and the relative position of the robot to the risk area.

[0040] According to an embodiment of the present invention, since the risk area can be adjusted accordingly based on the distance between the two hands, the robot can calculate the target repulsive force based on the distance and direction from the risk area using the artificial potential field method.

[0041] In operation S140, the target repulsion force is smoothly updated based on the actual repulsion force of the previous cycle to obtain the actual repulsion force of the current cycle; the actual repulsion force of the current cycle is vector-synthesized with the attraction force generated by the target point to obtain the resultant force on the robot.

[0042] When the relative positions of the operator's hands change, the currently effective risk area may also change. Since different risk areas have different spatial boundaries, there may be a difference between the target repulsive force determined based on the current risk area and the repulsive force actually acting on the robot in the previous cycle. Therefore, after obtaining the target repulsive force corresponding to the current cycle, it is not directly used as the actual repulsive force for the current cycle. Instead, the target repulsive force is smoothly updated by combining it with the actual repulsive force of the previous cycle, so that the actual repulsive force of the current cycle transitions from the actual repulsive force of the previous cycle to the target repulsive force.

[0043] Furthermore, based on the robot's current position and the target point's position, the gravitational force of the target point on the robot is determined, and the gravitational force is vector-synthesized with the actual repulsive force of the current cycle to obtain the resultant force currently acting on the robot. Here, the gravitational force is used to move the robot towards the target point, while the actual repulsive force is used to move the robot away from the currently active risk area. Thus, the resulting resultant force simultaneously reflects the robot's movement requirements of moving towards the target point and avoiding the risk area corresponding to the operator's hands.

[0044] For example, the position of the robot's end effector in space is set as q The target point location is q goal Gravity can be expressed as:

[0045] in, ε Let gravitational constant be ( q goal - q ) represents the position difference vector from the current position of the robot's end effector to the target point.

[0046] Therefore, the net force acting on the robot can be obtained. It can be represented as:

[0047] in, This represents the actual repulsive force during the current cycle.

[0048] By using the above method, when the risk area changes with the relative position of the operator's hands, the sudden change in repulsive force caused by the change in the risk area can be reduced, and the robot can perform the corresponding obstacle avoidance movement while approaching the target point.

[0049] When operating S150, the robot's direction of motion is determined based on the resultant force, and the robot is controlled to perform obstacle avoidance maneuvers.

[0050] According to an embodiment of the present invention, since the resultant force has been obtained, the direction of the robot's movement can be determined based on the resultant force, thereby enabling the robot to perform obstacle avoidance movements.

[0051] Through the above configuration, from the perspective of the repulsion formation mechanism, when the hands are close together, the joint risk area replaces two competing independent boundaries with a continuous envelope boundary, enabling the robot to obtain a stable outward normal repulsion force near the hands, reducing mutual cancellation, repeated superposition of repulsion forces, and repeated changes in the dominant direction. When the hands are far apart, the two independent risk areas are restored to retain a middle passage space that meets safety requirements and avoid unnecessary detours caused by the long-term existence of the joint envelope. During the risk area state adjustment process, when the distance between the hands is within the hysteresis interval, the state of the previous risk area is maintained to suppress frequent switching caused by detection noise or slight hand tremors. When the distance between the hands crosses the corresponding switching threshold, the current actual repulsion force is gradually updated to approach the target repulsion force after the switch through repulsion force smoothing, thereby reducing the repulsion force change within a single control cycle. Thus, dynamic risk area adjustment, hysteresis loop, and repulsion force smoothing update respectively achieve risk boundary matching, risk area state stabilization, and connection of control actions before and after switching, jointly improving the safety, stability, and motion continuity of the robot's obstacle avoidance process in dual-arm collaborative scenarios.

[0052] In one illustrative embodiment, when the risk area state is determined to be a first risk area state, the risk area is determined to be a spherical area; the center of the spherical area is the midpoint of the line connecting the first key point and the second key point.

[0053] According to an embodiment of the present invention, the radius of the spherical region is the distance from the center of the sphere to the outer edge of the more distant hand model, plus a preset safety margin; when the left and right hand models are the same size, it can be the sum of half the distance between the center points of the two hands, the radius of the hand model, and the safety margin. This setting enables the combined risk area to completely surround the hands and their gap, and reserves a safety space for positioning errors and dynamic movements.

[0054] Figure 5 A schematic diagram illustrating the risk area coverage of an example of the present invention is shown.

[0055] In one illustrative embodiment, when the risk area state is determined to be a second risk area state, a first ellipsoidal risk area surrounding the left arm is determined based on a first key point, a third key point, and a fifth key point, and a second ellipsoidal risk area surrounding the right arm is determined based on a second key point, a fourth key point, and a sixth key point.

[0056] like Figure 5 As shown, taking the right ellipsoidal risk area (corresponding to the right arm) as an example, the same applies to the left side. The key points of the human operator's arm are... P armR The calculated center point of the ellipsoid P centerE The direction of the extension of the line between them is set as x In the positive direction of the axis, the length 'a' of the semi-axis in this direction is equal to the Euclidean distance between the two points mentioned above; y shaft and z The axes are all perpendicular to x Axis, the corresponding semi-axis lengths b and c Equal. This setting ensures that the principal axes of the two ellipsoidal regions are aligned with the directions of the human operator's left and right arms, respectively, and fully cover the corresponding arms. The semi-axes b and c can also be determined based on the maximum equivalent radius and safety margin of the corresponding arm model to accommodate different operator body shapes and perception errors.

[0057] In one illustrative embodiment, the operation of determining the target repulsion force based on the distance between the left and right hands and the relative positional relationship between the robot and the risk area may include: determining a binary state factor based on the relationship between the distance between the left and right hands and a preset distance threshold; determining the currently effective risk area based on the value of the binary state factor; and determining the target repulsion force based on the relative positional relationship between the robot and the currently effective risk area.

[0058] According to an embodiment of the present invention, a binary state factor is used to characterize the current risk area state, and different values ​​correspond to different risk area states. By comparing the distance between the left and right hands with a preset distance threshold, the value of the binary state factor can be determined based on the current relative position of the hands, thereby making the risk area corresponding to the current relative position of the hands effective and avoiding different risk areas from participating in the determination of repulsive force in the current cycle at the same time.

[0059] After determining the currently active risk area, the obstacle avoidance effect of the risk area on the robot can be determined based on the robot's position relative to the currently active risk area, thereby obtaining the target repulsion force corresponding to the current cycle.

[0060] In one illustrative embodiment, the specific process of determining the binary state factor based on the relationship between the distance between the left and right hands and a preset distance threshold may include the following process: If the spacing is less than or equal to a preset spacing threshold, the binary state factor is determined as the first state value. If the spacing is greater than the preset spacing threshold but less than the sum of the preset spacing threshold and the hysteresis bandwidth, the value of the binary state factor from the previous period is determined as the value of the binary state factor for the current period. If the spacing is greater than or equal to the sum of the preset spacing threshold and the hysteresis bandwidth, the binary state factor is determined as the second state value.

[0061] For example, binary risk region state factor w k Its assignment can be performed according to the following rules:

[0062] in, δ This represents the hysteresis bandwidth. The hysteresis loop utilizes the binary state factor of the previous cycle. w k-1 Two different switching thresholds are established: when the distance between the hands decreases to a preset distance threshold. d min and below, w k Switch to 1 to enter the first risk zone state; when the distance between the hands increases to d min + δ At or above, w k Switch to 0 to enter the second risk zone state; when the distance between the hands is... d min and d min + δ In between, w k Maintain the value from the previous period. This hysteresis interval is used to lock in state fluctuations caused by subtle changes, avoiding frequent changes in the shape of risk areas due to image noise or slight hand movements.

[0063] According to an embodiment of the present invention, when the robot begins dynamic obstacle avoidance, if the initial hand-to-hand distance is within the hysteresis interval, the initial state factor is assumed to be... w 0=1, and joint risk areas are preferred to avoid potential risks caused by ambiguous status.

[0064] Furthermore, when the distance between the hands crosses the boundary of the hysteresis interval, the state factor... w k Switching between values ​​of 0 and 1 allows the joint risk region and the independent risk region to alternately take effect. Since the geometric boundaries of the two risk regions and their corresponding repulsive forces differ, directly using the repulsive force generated by the switched risk region might cause a sudden change in the repulsive force experienced by the robot's end effector at the switching moment. Therefore, a state factor is used...w k The repulsive force generated by the identified current effective risk area is used as the target repulsive force. Combined with the actual repulsive force applied in the previous control cycle, the actual repulsive force of the current control cycle is gradually updated to achieve a smooth transition of obstacle avoidance before and after the switch.

[0065] Figure 6 A schematic diagram of a repulsive field according to an example of the present invention is shown. For example... Figure 6 As shown in (a), the upper part corresponds to the state where the arms are close together (the spherical risk area covers the hands), and the lower part corresponds to the state where the arms are far apart (the ellipsoidal risk area completely covers the arms). Figure 6 (b) shows the repulsive field after reconstruction: when the two arms are close together, the repulsive field is formed by superposition of the two arm model and the risk area; when the two arms are far apart, the repulsive field is generated only based on the risk area.

[0066] In one illustrative embodiment, the process of determining the target repulsive force based on the relative positional relationship between the robot and the currently active risk area may include: Determine the shortest distance between the robot and the surface of the currently active risk area, as well as the direction of the repulsive force corresponding to the shortest distance.

[0067] Based on the shortest distance and the direction of the repulsive force, determine the repulsive force corresponding to the currently effective risk area, and based on the value of the binary state factor, determine the repulsive force corresponding to the current risk area state as the target repulsive force.

[0068] According to an embodiment of the present invention, the magnitude of the repulsive force can be determined based on an artificial repulsive field, as shown in the following formula:

[0069] in, σ It is the repulsive constant. ρ 0 represents the distance threshold affected by repulsive force. ρ i This is the shortest distance from the robot's end effector to the surface of the corresponding risk area.

[0070] Furthermore, repulsive force F repi Due to the negative gradient of the potential field (- U repi This is derived through rigorous derivation:

[0071] Wherein, unit vector e i = ρ i It strictly represents the gradient of distance, which geometrically corresponds to the outward normal vector of the obstacle surface.

[0072] It should be clarified that the shortest distance ρ i The calculation should be based on the dynamic risk region first. Therefore, in the generation of the repulsive field in the reconstruction, it is not necessary to define the distance between the hand and arm junction inside the double-arm geometric model. This simplifies the computational complexity in high dynamic environments and avoids the computational redundancy caused by local geometric interference in traditional methods.

[0073] Furthermore, it can be based on the binary state factor w k The candidate repulsive forces determined based on the two risk region states are mutually exclusive to obtain the target repulsive force for the current period. :

[0074] in, For the repulsive force in the first risk zone state, This refers to the repulsive force in the second risk zone state.

[0075] In one example, the joint risk region is in effect under the first risk region state. The shortest distance from the robot end effector to the surface of the joint risk region is determined, and the repulsive force direction corresponding to the shortest distance is determined. Based on the shortest distance and the repulsive force direction, the repulsive force generated by the joint risk region is determined according to the above repulsive force calculation method, and this force is used as the repulsive force corresponding to the first risk region state.

[0076] In another example, in the second risk area state, two independent risk areas corresponding to the left and right arms are activated, respectively. For each independent risk area, the shortest distance from the robot's end effector to the surface of that independent risk area and the repulsive force direction corresponding to the shortest distance are determined, and the repulsive force generated by that independent risk area is determined based on the corresponding shortest distance and repulsive force direction. Finally, the repulsive forces generated by the two independent risk areas are vector-synthesized to obtain the repulsive force corresponding to the second risk area state.

[0077] Therefore, within any control cycle, the repulsive force corresponding to the current effective risk area state can be obtained. Subsequently, based on the value of the binary state factor, a mutually exclusive selection is made between the repulsive forces corresponding to the first and second risk area states to obtain the target repulsive force for the current control cycle.

[0078] In one illustrative embodiment, the process of smoothly updating the target repulsion force based on the actual repulsion force of the previous cycle to obtain the actual repulsion force of the current cycle includes: weighting and summing the target repulsion force and the actual repulsion force of the previous cycle according to the repulsion force smoothing coefficient to obtain the actual repulsion force of the current cycle.

[0079] According to an embodiment of the present invention, in order to reduce the sudden change in actual repulsion force caused by risk area switching, the current target repulsion force is smoothly updated based on the actual repulsion force of the previous cycle to obtain the actual repulsion force of the current control cycle. :

[0080] in, This represents the actual repulsive force from the previous control cycle. α This is the repulsion smoothing coefficient. When 0 < α When <1, the current actual repulsive force gradually transitions between the actual repulsive force of the previous control cycle and the current target repulsive force; when α When the value is 1, the current actual repulsive force is directly taken as the target repulsive force to meet the needs of rapid obstacle avoidance response. In the initial control cycle, the previous actual repulsive force is initialized as the target repulsive force corresponding to the current initial risk area. After each control cycle ends, the actual repulsive force of the current control cycle is used as the previous actual repulsive force for the next control cycle, so that the actual repulsive force gradually approaches the target repulsive force in continuous control cycles.

[0081] Figure 7 A schematic diagram of a collaborative platform according to an example of the present invention is shown; Figure 8 The diagram shows a robot dynamic obstacle avoidance verification of an example of the present invention.

[0082] like Figure 7 As shown, in one example, a human operator performs corresponding operations according to the blue curve and preset steps; the collaborative robot, using the HITBOT Z-Arm S1400, is equipped with a Dahuan AG-160-95 electric gripper at its end, and collaborates along the path indicated by the green curve. An Intel RealSense D435i depth camera is positioned on the left side of the experimental platform to detect obstacles between the human operator's arms in real time.

[0083] like Figure 8 As shown, the initial path of the robot from the starting point to the target point is represented by a solid green line; the human operator's arms interfered with its original planned path multiple times, and the interfered sections are marked with dashed red lines; the dynamic obstacle avoidance path successfully achieved by the robot is represented by a solid blue line.

[0084] The six stages mentioned above are as follows: Phase 1 (Collaboration Begins): The robot picks up the part from the starting point along the original path and begins to move; Phase 2 (Arms Approaching): The human operator uses both arms to pick up the part located in the lower left corner and the tool located in the upper right corner of the collaboration area, respectively; Phase 3 (Avoiding the right arm): When the human operator's right arm retrieves the tool, it obstructs the robot's path, and the robot performs its first obstacle avoidance maneuver. Phase 4 (Avoiding Arms): The human operator's arms move close together and perform the assembly task, blocking the robot's path again. The robot then performs a second obstacle avoidance maneuver. Phase 5 (Avoiding the left arm): When the human operator puts the used tool back in place, it causes an obstruction, and the robot performs the third obstacle avoidance maneuver. Phase 6 (End of Collaboration): After successfully avoiding obstacles, the robot reaches the destination and places the parts.

[0085] Specifically, such as Figure 8 As shown in (a) and (b), the robot picks up a part from the starting point and moves along the initial path. The human operator uses both arms to pick up the part and the tool, respectively. During this process, the human operator's arms interfere with the robot's movement for the first time, preventing the robot from continuing along the original path. Figure 8 As shown in (c), the robot avoids the human operator's right arm during its first obstacle avoidance maneuver. Figure 8 As shown in (d), the human operator's arms move closer and begin the assembly task, creating a second obstacle for the robot after it has moved around. At this point, the robot treats the arms as a single obstacle and avoids it simultaneously. Figure 8 As shown in (e) and (f), after the assembly task is completed, the human operator's arms gradually separate. When the operator's left arm is placing the used tool, it obstructs the robot's movement for the third time. The robot successfully avoids the left arm and reaches the target position.

[0086] To further verify the effectiveness of the method of the present invention, this embodiment records the curves of the changes in the angles of each joint of the robot and the curves of the changes in the distance between the end effector and the two arms during the collaboration process.

[0087] Figure 9 The diagram shows the curves illustrating the changes in joint angles and obstacle avoidance distance of a robot according to an example of the present invention.

[0088] like Figure 9 As shown in (a), the joint angle change curves of the robot during the collaborative process exhibit three significant fluctuations, corresponding to three consecutive actions to avoid the human's arms. The second obstacle avoidance action takes slightly longer than the first two because the risk area strategy proposed in this invention treats the closely spaced arms as a single unit. This result demonstrates that the method of this invention can dynamically adapt to the approaching and separating movements of the human's arms, effectively avoiding potential collisions.

[0089] like Figure 9 As shown in (b), the real-time change in distance between the robot's end effector and the human's arms (hereinafter referred to as obstacle avoidance distance) remained above the preset safety threshold of 0.1 meters throughout the three obstacle avoidance maneuvers. The average obstacle avoidance distance was approximately 0.2 meters, indicating that the robot was able to maintain an obstacle avoidance distance above the preset safety threshold during the collaborative task.

[0090] Secondly, to verify the advantages of the dynamic risk area adjustment strategy in the method of the present invention when dealing with obstacles such as the arms of human operators, this embodiment compares the obstacle avoidance situation of the artificial potential field method before and after reconstruction.

[0091] Figure 10 A comparison chart of the usage of dynamic risk areas in embodiments of the present invention is shown.

[0092] like Figure 10 As shown, (a) and (b) illustrate the obstacle avoidance behavior using the artificial potential field method reconstructed according to the present invention. (a) ) and (b (c) and (c) demonstrate the obstacle avoidance behavior using the pre-reconstruction artificial potential field method. ) are (a) and (a) respectively This is a magnified view of a section of the image, used to demonstrate the repulsive forces near obstacles. In the image, the solid blue lines represent the robot's obstacle avoidance path using its two arms, and the dashed red lines represent path segments where physical collisions will occur.

[0093] The obstacle avoidance results comparison shows that when using the reconstructed artificial potential field method of this invention, the robot can successfully avoid arm obstacles and achieve safe cooperation regardless of whether the arms are close together or far apart. When using the unreconstructed artificial potential field method, physical collisions occur in all cases. Specifically, when the arms are close enough but a gap still exists, the traditional method determines this gap as a feasible path, causing the robot to pass through and collide. When the arms begin to separate, the traditional method also results in a collision when the robot approaches the other arm after avoiding one. This is because the traditional method lacks a dynamic risk area adjustment strategy, always treating the arms as two simple and independent dynamic obstacles. This result shows that activating the joint risk area when the arms are close together can directly close gaps that are actually insufficient for safe passage, preventing the robot from passing through incorrect areas. Restoring the two independent risk areas when the arms are far apart allows for separate responses to changes in the position of the left and right arms.

[0094] Furthermore, the repulsive field and repulsive calculation logic reconstructed in this invention can effectively avoid oscillations when the robot moves near its arms. For example... Figure 10 As shown in (c), the method of the present invention prioritizes the dynamic risk area adjustment strategy by reconstructing the generation and calculation logic of the repulsive force field. The robot does not traverse the area between the two arms, but instead smoothly navigates around the obstacles between the arms, thereby avoiding oscillations. Figure 10 (c As shown in the figure, when the artificial potential field method is used, when the robot moves between the two arms, a repulsive force field is generated near both arms. Under the combined action of the repulsive force, the robot's motion becomes unstable and oscillates.

[0095] The dynamic obstacle avoidance device provided by the present invention is described below. The robot dynamic obstacle avoidance device described below and the robot dynamic obstacle avoidance method described above can be referred to in correspondence.

[0096] Figure 11 A structural block diagram of a robot's dynamic obstacle avoidance device is shown.

[0097] like Figure 11 The robot dynamic obstacle avoidance device 1100 shown includes a key point acquisition module 1110, a risk state determination module 1120, a target repulsion force determination module 1130, a resultant force calculation module 1140, and an obstacle avoidance module 1150.

[0098] The key point acquisition module 1110 is used to acquire the position information of key points of the operator's two arms and construct a geometric model of the two arms based on the key points; wherein, the key points include a first key point and a second key point used to represent the operator's left hand and right hand, respectively.

[0099] The risk status determination module 1120 is used to determine the risk area status based on the relative positions of the first key point and the second key point; wherein, in the first risk area status, the risk area is a joint risk area surrounding the left hand, the right hand and the gap between them, and in the second risk area status, the risk area includes two independent risk areas surrounding the left arm and the right arm respectively.

[0100] The target repulsion force determination module 1130 is used to determine the target repulsion force based on the distance between the left and right hands and the relative positional relationship between the robot and the risk area.

[0101] The resultant force calculation module 1140 is used to smoothly update the target repulsive force based on the actual repulsive force of the previous cycle to obtain the actual repulsive force of the current cycle; and to perform vector synthesis of the actual repulsive force of the current cycle with the gravitational force generated by the target point to obtain the resultant force on the robot.

[0102] The obstacle avoidance module 1150 is used to determine the robot's direction of motion based on the resultant force and control the robot to perform obstacle avoidance movements.

[0103] Figure 12 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 12 As shown, the electronic device may include a processor 1210, a communications interface 1220, a memory 1230, and a communication bus 1240. The processor 1210, communications interface 1220, and memory 1230 communicate with each other via the communication bus 1240. The processor 1210 can call logic instructions from the memory 1230 to execute a robot dynamic obstacle avoidance method.

[0104] Furthermore, the logical instructions in the aforementioned memory 1230 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0105] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0106] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0107] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic obstacle avoidance method for robots, characterized in that, include: The location information of key points of the operator's two arms is obtained, and a geometric model of the two arms is constructed based on the key points; wherein, the key points include a first key point and a second key point used to represent the operator's left hand and right hand, respectively; The risk area status is determined based on the relative positions of the first key point and the second key point; wherein, in the first risk area status, the risk area is a joint risk area surrounding the left hand, the right hand and the gap between them, and in the second risk area status, the risk area includes two independent risk areas surrounding the left arm and the right arm respectively. The target repulsive force is determined based on the distance between the left and right hands and the relative position of the robot to the risk area; The target repulsive force is smoothly updated based on the actual repulsive force of the previous cycle to obtain the actual repulsive force of the current cycle; the actual repulsive force of the current cycle is vector-synthesized with the gravitational force generated by the target point to obtain the resultant force on the robot. The robot's direction of motion is determined based on the resultant force, and the robot is controlled to perform obstacle avoidance maneuvers.

2. The robot dynamic obstacle avoidance method according to claim 1, characterized in that, Determining the risk area status based on the relative positions of the first key point and the second key point includes: If the distance between the left and right hands is less than or equal to a preset distance threshold, the risk area state is determined to be the first risk area state. If the distance between the left and right hands is greater than the preset distance threshold, the risk area state is determined to be the second risk area state.

3. The robot dynamic obstacle avoidance method according to claim 2, characterized in that, Also includes: When the risk area status is determined to be the first risk area status, the risk area is determined to be a spherical area; The center of the spherical region is the midpoint of the line connecting the first key point and the second key point.

4. The robot dynamic obstacle avoidance method according to claim 2 or 3, characterized in that, The key points also include a third key point and a fourth key point for characterizing the operator's left wrist and right wrist, respectively, and a fifth key point and a sixth key point for characterizing the end of the operator's left forearm and right forearm. The method further includes: When the risk area state is determined to be the second risk area state, a first ellipsoidal risk area surrounding the left arm is determined based on the first key point, the third key point, and the fifth key point, and a second ellipsoidal risk area surrounding the right arm is determined based on the second key point, the fourth key point, and the sixth key point.

5. The robot dynamic obstacle avoidance method according to claim 2, characterized in that, The determination of the target repulsive force based on the distance between the left and right hands and the relative position of the robot to the risk area includes: The binary state factor is determined based on the relationship between the distance between the left and right hands and the preset distance threshold. The currently active risk area is determined based on the value of the binary state factor, and the target repulsive force is determined based on the relative positional relationship between the robot and the currently active risk area.

6. The robot dynamic obstacle avoidance method according to claim 5, characterized in that, The step of determining the binary state factor based on the relationship between the distance between the left and right hands and the preset distance threshold includes: If the spacing is less than or equal to the preset spacing threshold, the binary state factor is determined to be the first state value; If the spacing is greater than the preset spacing threshold and less than the sum of the preset spacing threshold and the hysteresis bandwidth, the value of the binary state factor of the previous period is determined as the value of the binary state factor of the current period. If the spacing is greater than or equal to the sum of the preset spacing threshold and the hysteresis bandwidth, the binary state factor is determined as the second state value.

7. The robot dynamic obstacle avoidance method according to claim 5, characterized in that, Determining the target repulsive force based on the relative positional relationship between the robot and the currently active risk area includes: Determine the shortest distance between the robot and the surface of the currently active risk area, and the direction of the repulsive force corresponding to the shortest distance; Based on the shortest distance and the direction of the repulsive force, the repulsive force corresponding to the currently effective risk area is determined, and based on the value of the binary state factor, the repulsive force corresponding to the current risk area state is determined as the target repulsive force.

8. The robot dynamic obstacle avoidance method according to claim 1, characterized in that, The step of smoothly updating the target repulsion force based on the actual repulsion force of the previous cycle to obtain the actual repulsion force of the current cycle includes: Based on the repulsion smoothing coefficient, the target repulsion force is weighted and summed with the actual repulsion force of the previous cycle to obtain the actual repulsion force of the current cycle.

9. A dynamic obstacle avoidance device for robots, characterized in that, include: The key point acquisition module is used to acquire the position information of key points of the operator's two arms and construct a geometric model of the two arms based on the key points; wherein, the key points include a first key point and a second key point used to represent the operator's left hand and right hand respectively; The risk status determination module is used to determine the risk area status based on the relative positions of the first key point and the second key point; wherein, in the first risk area status, the risk area is a joint risk area surrounding the left hand, the right hand and the gap between them, and in the second risk area status, the risk area includes two independent risk areas surrounding the left arm and the right arm respectively. The target repulsion force determination module is used to determine the target repulsion force based on the distance between the left and right hands and the relative positional relationship between the robot and the risk area. The resultant force calculation module is used to smoothly update the target repulsive force based on the actual repulsive force of the previous cycle to obtain the actual repulsive force of the current cycle; and to perform vector synthesis of the actual repulsive force of the current cycle with the gravitational force generated by the target point to obtain the resultant force on the robot. The obstacle avoidance module is used to determine the robot's direction of motion based on the resultant force and control the robot to perform obstacle avoidance movements.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the robot dynamic obstacle avoidance method as described in any one of claims 1 to 8.

Images