An obstacle avoidance method, device, apparatus and storage medium

Abstract

The specification discloses an obstacle avoidance method, device, equipment and storage medium, constructs a point cloud map according to point cloud data of an obstacle sent by an acquisition device, labels an obstacle area on the point cloud map, and then plans a motion trajectory of a mechanical arm based on the obstacle area. When the mechanical arm moves along the motion trajectory, sensor data sent by a collision sensor is monitored. When it is determined that the mechanical arm collides with the obstacle, a collision position at which the mechanical arm collides with the obstacle is determined according to the sensor data, and the obstacle area is updated based on the determined collision position. It can be seen that even if the mechanical arm collides with the obstacle during the movement of the mechanical arm, the collision position can be determined based on the sensor data obtained by the collision sensor at the time of collision, and the original obstacle area is updated, so as to re-plan the trajectory based on the updated obstacle area, and improve the accuracy of collision prevention in the trajectory planning of the mechanical arm.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to an obstacle avoidance method, apparatus, device, and storage medium. Background Technology

[0002] With the continuous progress in robotics research and technological development, robotics technology has been widely applied in various computer technology fields. Among them, robotic arms are an important component of the robotics field. Initially, robotic arms could only move according to fixed routes. Later, with the development of machine vision, machine vision technology can be used to collect the position information of obstacles in advance, and then plan the movement trajectory of the robotic arm to avoid the obstacles.

[0003] However, due to the limited accuracy of the obstacle location information acquisition, the planned movement trajectory of the robotic arm is not accurate enough, which leads to the robotic arm colliding with the obstacle when moving along the movement trajectory. Summary of the Invention

[0004] This specification provides an obstacle avoidance method, apparatus, device, and storage medium to partially solve the aforementioned problems existing in the prior art.

[0005] The following technical solution is adopted in this specification:

[0006] This manual provides an obstacle avoidance method, including:

[0007] Based on the point cloud data of each obstacle sent by the acquisition device, a point cloud map is constructed, and the obstacle areas are marked in the point cloud map;

[0008] Based on the obstacle area, plan the movement trajectory of the robotic arm, and make the robotic arm move according to the movement trajectory;

[0009] Monitor the sensor data sent by the collision sensor configured on the robotic arm, and determine whether the robotic arm has collided with an obstacle based on the sensor data;

[0010] When it is determined that the robotic arm collides with the obstacle, the collision position of the robotic arm and the obstacle is determined;

[0011] The obstacle area is updated based on the collision location, and the movement trajectory of the robotic arm is replanned based on the updated obstacle area, so that the robotic arm moves according to the replanned movement trajectory.

[0012] Optionally, determining whether the robotic arm has collided with an obstacle based on the sensor data specifically includes:

[0013] Determine whether the sensor data is greater than a preset safety threshold;

[0014] If not, no control command is sent, and the robotic arm continues to move along the motion trajectory;

[0015] If so, send a control command to the robotic arm to move it away from the obstacle.

[0016] Optionally, when it is determined that the robotic arm has collided with the obstacle, the collision position between the robotic arm and the obstacle is determined, specifically including:

[0017] Obtain the time of receiving the sensor data;

[0018] Based on the motion trajectory, the position of the robotic arm at the receiving moment is determined as the desired position of the robotic arm;

[0019] Based on the sensor data, the offset vector of the robotic arm on the motion trajectory is determined;

[0020] Based on the desired position of the robotic arm on the motion trajectory and the offset vector of the robotic arm on the motion trajectory, the collision position where the robotic arm collides with the obstacle is determined.

[0021] Optionally, determining the offset vector of the robotic arm on the motion trajectory specifically includes:

[0022] Obtain the impedance stiffness corresponding to each preset direction;

[0023] Determine whether the direction of the sensor data coincides with each of the preset directions;

[0024] If so, the offset vector of the robotic arm on the motion trajectory is determined based on the impedance stiffness corresponding to the coincident preset direction and the sensor data.

[0025] If not, the offset vector of the robotic arm on the motion trajectory is determined based on the preset directions and the sensor data.

[0026] Optionally, based on the preset directions and the sensor data, the offset vector of the robotic arm on the motion trajectory is determined, specifically including:

[0027] Based on the sensor data, determine the data components of the sensor data in each preset direction;

[0028] For each preset direction, the offset vector of the robotic arm in that preset direction is determined based on the predetermined impedance stiffness corresponding to that preset direction and the data components in that preset direction.

[0029] The offset vector of the robotic arm on the motion trajectory is determined based on the offset vector of the robotic arm in each preset direction.

[0030] Optionally, the obstacle region is updated based on the determined collision location, specifically including:

[0031] Based on the collision location, determine the distance between the collision location and each point cloud data contained in the point cloud map, and determine the target point from each point cloud data based on the determined distance;

[0032] Based on the collision location and the target point, determine the target obstacle area that includes the collision location;

[0033] Update the obstacle region based on the target obstacle region.

[0034] Optionally, based on the collision location and the target point, a target obstacle region including the collision location is determined, specifically including:

[0035] Using the collision location as the center of the sphere and the distance between the collision location and the target point as the radius of the sphere, a spherical region is constructed as the target obstacle region containing the collision location.

[0036] Optionally, the collision sensor is configured at the end of the robotic arm and the joint mechanism;

[0037] Monitoring the sensor data sent by the collision sensors configured on the robotic arm, and determining whether the robotic arm has collided with an obstacle based on the sensor data, specifically includes:

[0038] When the sensor data sent by the collision sensor configured on the end effector of the robotic arm exceeds a preset safety threshold, it is determined that the end effector of the robotic arm has collided with the obstacle; and / or

[0039] When the sensor data sent by the collision sensor configured on the joint structure of the robotic arm is greater than a preset safety threshold, it is determined that the joint mechanism of the robotic arm has collided with the obstacle.

[0040] This manual provides an obstacle avoidance device, including:

[0041] The obstacle area determination module is used to construct a point cloud map based on the point cloud data of each obstacle sent by the acquisition device, and to mark the obstacle area in the point cloud map;

[0042] The planning module is used to plan the motion trajectory of the robotic arm based on the obstacle area, so that the robotic arm moves according to the motion trajectory;

[0043] The judgment module is used to monitor the sensor data sent by the collision sensor configured on the robotic arm, and determine whether the robotic arm has collided with an obstacle based on the sensor data.

[0044] The collision location determination module is used to determine the collision location between the robotic arm and the obstacle when it is determined that the robotic arm has collided with the obstacle.

[0045] An update module is used to update the obstacle area based on the collision location, and to replan the motion trajectory of the robotic arm based on the updated obstacle area, so that the robotic arm moves according to the replanned motion trajectory.

[0046] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the obstacle avoidance method described above.

[0047] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the obstacle avoidance method described above.

[0048] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0049] This method constructs a point cloud map based on point cloud data of obstacles sent by the acquisition device, marks obstacle areas on the point cloud map, and then plans the movement trajectory of the robotic arm based on the obstacle areas. As the robotic arm moves along the movement trajectory, it monitors the sensor data sent by the collision sensor. When a collision between the robotic arm and an obstacle is determined, the collision position is determined based on the sensor data, and the obstacle area is updated based on the determined collision position. Therefore, even if the robotic arm collides with an obstacle during its movement, the collision position can be determined based on the sensor data acquired at the time of the collision, and the original obstacle area can be updated. This allows for a re-planning of the trajectory based on the updated obstacle area, improving the accuracy of collision avoidance in the robotic arm trajectory planning. Attached Figure Description

[0050] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0051] Figure 1 This is a flowchart illustrating one obstacle avoidance method described in this specification.

[0052] Figure 2 This is a schematic diagram of an obstacle avoidance system described in this specification;

[0053] Figure 3 This is a flowchart illustrating one obstacle avoidance method described in this specification.

[0054] Figure 4A This is a schematic diagram of a robotic arm used in this specification;

[0055] Figure 4B This is a schematic diagram of a robotic arm used in this specification;

[0056] Figure 5 This is a schematic diagram of an obstacle avoidance device provided in this specification;

[0057] Figure 6 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0059] Additionally, it should be noted that all actions involving the acquisition of signals, information, or data in this manual are performed in accordance with the relevant data protection laws and regulations of the country where the device is located, and with authorization from the owner of the relevant device.

[0060] Generally, robots have been applied in many fields, but robots inevitably encounter obstacles during operation, which makes it necessary for robots to avoid obstacles.

[0061] Take its application in the delivery field as an example. Robots that perform delivery tasks in the delivery field are usually unmanned vehicles. During the delivery process, there may be situations where other vehicles or pedestrians suddenly appear in front of the unmanned vehicle. This situation mostly occurs in scenarios where there is not enough time to plan a path to avoid the obstacle. Therefore, it is necessary to control the speed of the unmanned vehicle. Otherwise, due to excessive speed and kinetic energy, the unmanned vehicle may collide with the obstacle and cause significant damage, posing a safety hazard.

[0062] Taking its application in the medical field as an example, the emergence of surgical robots aligns with the development trend of precision surgery. Surgical robots have become powerful tools to assist doctors in performing surgeries, designed to use minimally invasive techniques to precisely execute complex surgical procedures. Because surgical robots cause less harm to patients, resulting in less bleeding and faster recovery, they significantly shorten postoperative hospital stays and substantially improve postoperative survival and recovery rates. As a high-end medical device, they are now widely used in various clinical surgeries. Therefore, how to precisely plan the motion trajectory of the robotic arm and utilize it to complete delicate surgical operations has become an urgent problem to be solved.

[0063] Obstacle detection is particularly important in robotic arm trajectory planning. Obstacle detection can utilize acquisition devices such as LiDAR or depth cameras to pre-collect point cloud data of obstacles in the surgical environment. Then, a point cloud map is constructed based on this data. This map contains several point clouds that indicate the location of obstacles. By labeling obstacles in the constructed point cloud map, marked obstacle areas can be identified. During robotic arm path planning, a trajectory that avoids these obstacle areas is planned, ensuring that the robotic arm does not collide with obstacles during movement in the surgical environment.

[0064] However, trajectory planning based on obstacle regions marked with pre-collected point cloud information may have the following problems: First, the accuracy of obstacle region marking depends on the precision of the point cloud data collected by the acquisition device. If the acquisition device has low precision, there may be situations where obstacles actually exist on the planned robotic arm's trajectory due to incomplete point cloud data collection. Second, during the surgery, unexpected obstacle movement may occur, leading to changes in the obstacle region during the operation.

[0065] All of the above issues could potentially lead to obstacles actually existing in the pre-planned movement trajectory of the robotic arm.

[0066] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0067] Figure 1 This is a flowchart illustrating one obstacle avoidance method provided in this specification.

[0068] S100: Construct a point cloud map based on the point cloud data of each obstacle sent by the acquisition device, and mark the obstacle areas in the point cloud map.

[0069] In the embodiments of this specification, the executing entity can be a server, which can be located in a device independent of the robotic arm. In one or more embodiments of this specification, the robotic arm can perform responsive tasks in any existing environment, such as performing order delivery tasks in a delivery environment, performing surgical assistance tasks in a surgical environment, or performing industrial parts gripping tasks in an industrial environment. For ease of understanding, this specification uses the application of a robotic arm in a surgical environment as an example to illustrate the specific technical solution.

[0070] Typically, a surgical environment includes an operating table. In surgical scenarios where a robotic arm performs surgery on a patient, the patient sits on the operating table while the robotic arm carries surgical instruments to perform the procedure according to a predetermined surgical position. Therefore, during the movement of the robotic arm from its initial position to the predetermined surgical position within the surgical environment, it is necessary to avoid collisions with the operating table, the patient on the operating table (excluding the predetermined surgical position where the patient needs to undergo surgery), and the robotic arm itself to prevent surgical hazards. Obstacles include at least the patient, the operating table, and the robotic arm. The point cloud data of each obstacle can represent the position of each point on the obstacle surface in a three-dimensional coordinate system. Of course, in actual surgical environments, other surgical equipment can also be set up, such as human body scanning equipment like CT scanners, display devices for displaying scanned images, and robotic arms for performing other surgical operations; this manual does not limit these.

[0071] The acquisition device can be a point cloud acquisition device such as a depth camera or LiDAR set up in the surgical environment, or it can be mounted on a robotic arm. By capturing several images of the surgical environment, point cloud data of each obstacle in the surgical environment is pre-acquired. The acquisition device can be located in a device independent of the robotic arm within the surgical environment. In the embodiments of this specification, to comprehensively acquire point cloud data of each obstacle in the surgical environment, one or more acquisition devices can be deployed in the surgical environment. Furthermore, to ensure that the point cloud data acquired by the acquisition device covers as many obstacles in the surgical environment as possible, the acquisition device can perform at least one point cloud data acquisition process; this specification does not limit this approach.

[0072] Furthermore, since the data acquisition devices for collecting point cloud data of various obstacles are typically installed in fixed locations within the surgical environment, the map coordinate system of the point cloud map can be transformed to the world coordinate system based on the installation location of the acquisition devices. This allows the robotic arm to move according to a motion trajectory planned based on the point cloud map. Additionally, in one or more embodiments of this specification, the constructed point cloud map can be categorized according to the type of obstacle, and obstacle areas and passable areas can be marked based on the categorized obstacles. This ensures that the planned trajectory avoids overlapping with obstacle areas during subsequent trajectory planning, thereby preventing collisions between the robotic arm and obstacles.

[0073] S102: Based on the obstacle area, plan the motion trajectory of the robotic arm so that the robotic arm moves according to the motion trajectory.

[0074] Specifically, the surgical location where the patient needs to undergo surgery and the initial position of the robotic arm can be determined in advance. Based on the initial position, the surgical location, and the obstacle areas marked on the point cloud map, the motion trajectory of the robotic arm is planned. This trajectory can indicate the three-dimensional position, velocity, and acceleration of the robotic arm at different moments during its movement from the initial position to the surgical location. Normally, the planned trajectory can avoid the obstacle areas marked on the point cloud map. However, due to the limited accuracy of the data acquisition equipment and the possibility of unexpected movement of obstacles during the surgery, the accuracy of the obstacle areas marked on the point cloud map is low, resulting in low accuracy of the planned trajectory based on these obstacle areas.

[0075] S104: Monitor the sensor data sent by the collision sensor configured on the robotic arm, and determine whether the robotic arm has collided with an obstacle based on the sensor data.

[0076] In practical applications, due to the limited accuracy of the acquisition equipment or the movement of pre-positioned obstacles during surgery, unexpected obstacles may appear on the robotic arm's trajectory. In this case, the robotic arm, moving along its pre-planned trajectory, may collide with the obstacle. The collision detection can be determined based on monitored sensor data. The collision sensor used for collision detection can be a force / torque sensor, an acceleration sensor, or other sensors capable of real-time collision detection. Alternatively, a collision detection method using an external force observer model can be employed. This manual uses impedance force data detected by the collision sensor as an example to illustrate the specific solution. The collision sensor can be placed at the end of the robotic arm, at one of its joints, or both at the end of the robotic arm and at its joints to handle collisions with obstacles near or far from the trajectory.

[0077] Specifically, the sensor data sent by the collision sensor can be in the form of a sequence. For example, in chronological order, the magnitude and direction of the resistance force experienced by the robotic arm at each moment transmitted by the collision sensor can be recorded to form a sensor data sequence. By judging the relationship between the magnitude of the resistance force of the robotic arm at each moment in the sensor data sequence and a preset safety threshold, the collision situation between the robotic arm and the obstacle at each moment can be determined.

[0078] For example, if the preset safety threshold is 10N, the magnitudes of the resistance force experienced by the robotic arm at the first, second, and third moments are 0N, 11N, and 4N, respectively. Based on the relationship between the magnitude of the resistance force at each moment and the preset safety threshold, it can be determined that the robotic arm collided with the obstacle at the second moment.

[0079] Of course, it is understandable that the collision sensors mounted on the robotic arm can also send sensor data when they acquire it. The timing of sensor data transmission can be determined based on the specific application scenario, and this manual does not impose any restrictions on it.

[0080] S106: When it is determined that the robotic arm collides with the obstacle, determine the collision position of the robotic arm and the obstacle.

[0081] When a collision between the robotic arm and the obstacle is detected, the robotic arm can first be controlled according to a pre-set motion strategy, such as terminating the ongoing movement task, implementing an emergency braking strategy, or escaping the obstacle area. This is to prevent further damage to surgical equipment, the patient, and the robotic arm itself caused by the collision.

[0082] When the robotic arm moves along the planned trajectory, a collision occurs with an obstacle, indicating that the pre-planned trajectory did not precisely avoid the obstacle area in the point cloud map. This could be due to two reasons: either the obstacle area in the point cloud map has limited accuracy, or the obstacle in the surgical environment has shifted during the procedure. To obtain the position of the obstacle that collided with it after the collision, the collision location can be used as the new location of the obstacle and updated in the obstacle area of ​​the point cloud map.

[0083] S108: Update the obstacle area according to the collision location; and replan the motion trajectory of the robotic arm according to the updated obstacle area, so that the robotic arm moves according to the replanned motion trajectory.

[0084] In this step, the determined collision location is added to the obstacle area, achieving the goal of updating the obstacle position in real time during the operation. Even if only one point cloud data acquisition is performed by the acquisition device before the operation, the obstacle position can be updated in real time through the sensor data sent by the collision sensor on the robotic arm. This solves the problem of collision between the robotic arm and the obstacle due to the limited accuracy of point cloud data acquisition and the displacement of the obstacle during the operation.

[0085] based on Figure 1The obstacle avoidance method shown constructs a point cloud map based on point cloud data of obstacles sent by a data acquisition device, marks obstacle areas on the point cloud map, and then plans the movement trajectory of the robotic arm based on the obstacle areas. As the robotic arm moves along the trajectory, it monitors sensor data sent by the collision sensor. When a collision between the robotic arm and an obstacle is determined, the collision location is determined based on the sensor data, and the obstacle area is updated based on the determined collision location. Therefore, even if the robotic arm collides with an obstacle during surgery, the collision location can be determined based on the sensor data acquired at the time of the collision, and the original obstacle area can be updated. This allows for a re-planning of the trajectory based on the updated obstacle area, improving the accuracy of collision avoidance in the robotic arm trajectory planning.

[0086] Based on the above obstacle avoidance method, this specification also provides an obstacle avoidance system, which includes a data acquisition device, a control device, and a robotic arm; wherein, a collision sensor is configured on the robotic arm.

[0087] The acquisition device is used to pre-acquire point cloud data of each obstacle and send the point cloud data of each obstacle to the control device;

[0088] The control device is configured to construct a point cloud map based on point cloud data of each obstacle sent by a data acquisition device, and mark obstacle areas in the point cloud map; plan the motion trajectory of the robotic arm based on the obstacle areas, and send the motion trajectory to the robotic arm; when receiving sensor data sent by a collision sensor configured on the robotic arm, determine whether the robotic arm has collided with an obstacle based on the sensor data; when it is determined that the robotic arm has collided with the obstacle, determine the collision position; update the obstacle area based on the collision position, and re-plan the motion trajectory of the robotic arm based on the updated obstacle area, and send the re-planned motion trajectory to the robotic arm;

[0089] The robotic arm is used to move according to the motion trajectory. When the robotic arm collides with an obstacle, the collision sensor configured on the robotic arm acquires sensor data and sends the sensor data to the control device. When a replanned motion trajectory is received, the robotic arm moves according to the replanned motion trajectory.

[0090] As can be seen, by introducing a passive collision avoidance obstacle detection strategy, after the robotic arm comes into contact with an obstacle, it acquires sensor data through the collision sensors configured on the robotic arm to perform collision detection. When it is determined that the robotic arm has collided with the obstacle, the collision position between the robotic arm and the obstacle is determined through the sensor data, and then the obstacle area in the point cloud map is updated based on the collision position in order to replan the movement trajectory of the robotic arm.

[0091] like Figure 2 As shown, the obstacle avoidance system includes a robotic arm 1, a control device 2, and a data acquisition device 3. The robotic arm 1 is equipped with a collision sensor 11. In one or more embodiments of this specification, an auxiliary robotic arm 4 may also be included to assist the surgical operation of the robotic arm 1. The robotic arm 1 and the auxiliary robotic arm 4 may be located on different devices on either side of the operating table. The control device 2 may be located on other devices independent of the devices where the robotic arm 1 and the auxiliary robotic arm 4 are located; this specification does not limit this. Furthermore, the control device 2 in the obstacle avoidance system provided in the embodiments of this specification is mainly used to construct a point cloud map and mark obstacle areas, as well as to analyze sensor data sent by the collision sensor. Optionally, sub-control devices performing the above two functions may be located in different devices to perform control tasks separately.

[0092] In the surgical environment, robotic arm 1 and auxiliary robotic arm 4 can be mounted on different devices, on the operating table 5, or integrated into the same device; this specification does not limit this. The data acquisition device 3 can be a lidar or depth camera fixed within the surgical environment. Alternatively, the data acquisition device 3 can be mounted on robotic arm 1 (not shown in the figure). In the embodiments described in this specification, it is necessary to plan the movement trajectory of the robotic arm to avoid obstacles. The obstacles mentioned here refer to robotic arm 1 and may include, for example... Figure 2 The surgical environment shown includes the robotic arm 1 itself, the control device 2, the data acquisition device 3, the auxiliary robotic arm 4, the operating table 5, and the patient 6. Of course, the type and number of obstacles can be determined according to the specific surgical scenario, and this manual does not limit this.

[0093] In the embodiments described in this specification, such as Figure 1 Steps S104 to S106 monitor the sensor data sent by the collision sensor configured on the robotic arm, and determine whether the robotic arm has collided with an obstacle based on the sensor data. Then, when it is determined that the robotic arm has collided with the obstacle, the collision position is determined. This can be achieved in the following ways: Figure 3 As shown:

[0094] S200: Based on the sensor data, determine whether the sensor data exceeds a preset safety threshold. If yes, proceed to step S204; otherwise, proceed to step S202.

[0095] S202: No control command is sent, so that the robotic arm continues to move along the motion trajectory.

[0096] When the sensor data is not greater than the preset safety threshold, it indicates that the obstacle may be relatively soft, such as medical cotton. It is also possible that the robotic arm will only scrape against the obstacle. In this case, even if the robotic arm continues to move along the originally planned motion trajectory, it will not cause damage to the obstacle or the robotic arm. Therefore, no control command needs to be sent, and the robotic arm can continue to move along the original motion trajectory.

[0097] S204: Send a control command to the robotic arm to move the robotic arm away from the obstacle.

[0098] When the sensor data exceeds a preset safety threshold, it indicates that the obstacle may be relatively hard, resulting in significant resistance to the robotic arm upon collision. In this case, a control command can be sent to the robotic arm to alter its trajectory and move away from the obstacle. For example, it can retreat from the current collision position along a pre-planned trajectory, retreat a predetermined distance in a predetermined direction, or return to its original position. Of course, the specific hovering position of the robotic arm after leaving the collision location can be determined based on the specific surgical scenario; this manual does not impose any limitations on this.

[0099] S206: Obtain the impedance stiffness corresponding to each preset direction, and determine whether the direction of the sensor data coincides with each preset direction; if yes, proceed to step S208, otherwise proceed to step S210.

[0100] Typically, when a robotic arm comes into contact with an obstacle in the surgical environment, impedance is used to describe the characteristics of the robotic arm. The relationship between the contact force and position between the robotic arm's end effector and the obstacle is adjusted by regulating the stiffness coefficient of the impedance controller. In the embodiments of this specification, multiple impedance stiffnesses can be preset, and these stiffnesses have directions. During the impedance movement of the robotic arm, the actual position of the robotic arm may differ from the desired position under the influence of the obstacle. To determine the collision location between the robotic arm and the obstacle, sensor data sent by a collision sensor is needed to determine the offset vector of the robotic arm in each preset direction. This sensor data can be impedance force data, which may include the magnitude and direction of the impedance force.

[0101] S208: Based on the impedance stiffness corresponding to the coincident preset direction and the sensor data, determine the offset vector of the robotic arm on the motion trajectory. Execute step S216.

[0102] If the direction of the resistance force coincides with the preset direction, the offset vector of the robotic arm in the direction of the resistance force can be determined directly based on the ratio of the resistance force to the resistance stiffness corresponding to the preset direction. This is the offset vector of the robotic arm on the motion trajectory.

[0103] For example, such as Figure 4AThe diagram illustrates the force situation at the end effector of a robotic arm. The collision sensor at the end effector acquires data showing an impedance force F1, with the direction as shown in the figure. Since the direction of the impedance force F1 coincides with the direction of the preset impedance stiffness K1, the offset vector ΔP1 of the robotic arm along its trajectory can be determined based on the ratio of the impedance force F1 to the preset impedance stiffness K1. This offset vector is then applied to the desired position P of the robotic arm. des At that time, it can be based on the desired position P des The collision position between the robotic arm and the obstacle is determined by the offset vector ΔP1. cur .

[0104] S210: Determine the data components of the sensor data in each preset direction based on the sensor data.

[0105] If the direction of the impedance force does not coincide with any of the preset directions, the impedance force can be decomposed to obtain the impedance component force in each preset direction.

[0106] S212: For each preset direction, determine the offset vector of the robotic arm in that preset direction based on the predetermined impedance stiffness corresponding to that preset direction and the data components in that preset direction.

[0107] Furthermore, in each preset direction, the offset vector of the robotic arm in each preset direction is determined based on the ratio between the impedance component and the impedance stiffness.

[0108] S214: Determine the offset vector of the robotic arm in the direction of its motion trajectory based on the offset vectors of the robotic arm in each preset direction.

[0109] The offset vectors of the robotic arm in each preset direction are synthesized to obtain the offset vector of the robotic arm in the direction of its motion trajectory. The decomposition of force and the synthesis of offset vectors both follow existing decomposition and synthesis methods, and this specification does not impose any limitations on them.

[0110] For example, such as Figure 4B The figure shows the force situation at the end effector of a robotic arm. The collision sensor at the end effector acquires sensor data showing an impedance force F2, the direction of which is shown in the figure. Since the direction of the impedance force F2 does not coincide with any preset directions, the impedance force F2 is decomposed along the preset directions X and Y to obtain the impedance component F in the X direction. x The impedance component Fy in the Y direction is determined by the impedance stiffness K in the X direction. x and the impedance stiffness K in the Y direction yThe offset vectors Δx and Δy of the robotic arm in the X and Y directions are determined respectively. Then, based on the offset vectors Δx and Δy, the synthesized offset vector ΔP2 is determined as the offset vector of the robotic arm in the direction of its motion trajectory. Given the desired position P of the robotic arm... des At that time, it can be based on the desired position P des The collision position between the robotic arm and the obstacle is determined by the offset vector ΔP2. cur .

[0111] S216: Obtain the receiving time of the sensor data, and determine the position of the robotic arm at the receiving time based on the motion trajectory, as the desired position of the robotic arm.

[0112] Collision sensors mounted on the robotic arm acquire data when it collides with an obstacle. Typically, the collision sensors immediately send this data to a server, which then uses this data, along with preset safety thresholds, to determine whether the collision significantly impacted the robotic arm. Because a collision prevents the robotic arm from following its planned trajectory, its real-time position deviates; in other words, the robotic arm fails to reach the desired position on its trajectory at a given moment.

[0113] Specifically, the server can obtain the moment when it receives sensor data and, based on the motion trajectory, determine the position of the robotic arm at the moment of reception if it does not collide with the obstacle, and use this position as the desired position.

[0114] S218: Determine the collision position where the robotic arm collides with the obstacle based on the desired position of the robotic arm and the offset vector of the robotic arm in the direction of its motion trajectory.

[0115] Specifically, by determining the desired position on the trajectory and the offset vector of the robotic arm in the direction of the trajectory, the collision point between the robotic arm and the obstacle can be determined. This collision point is added to the obstacle area in the point cloud map. Without requiring secondary point cloud acquisition during surgery, the positions of obstacles in the surgical environment are updated in real time, making the obstacle area in the point cloud map more accurate and thus improving the obstacle avoidance accuracy of the robotic arm's trajectory.

[0116] Based on such Figure 3The proposed scheme decomposes the impedance force obtained by the collision sensor when the robotic arm collides with the obstacle in each preset direction to obtain the impedance component force in each preset direction. Based on the impedance stiffness in each preset direction, the offset vector of the robotic arm in each preset direction is determined by using the ratio of the impedance component force to the impedance stiffness in each preset direction. By synthesizing the offset vectors in each preset direction, the distance difference between the actual position and the desired position of the robotic arm can be obtained. Then, combined with the desired position of the robotic arm on the motion trajectory, the collision position when the robotic arm collides with the obstacle can be obtained.

[0117] In the embodiments described in this specification, such as Figure 1 Step S108 updates the obstacle area based on the determined collision location so that the motion trajectory of the robotic arm can be replanned based on the updated obstacle area. This can be achieved in the following ways.

[0118] First, based on the collision location, the distance between the collision location and each point cloud data contained in the point cloud map is determined, and based on the determined distance, the target point is determined from each point cloud data.

[0119] Typically, because the collision location is outside the obstacle area marked on the point cloud map, the planned trajectory, when pre-planning the trajectory based on the obstacle area, does not avoid the collision location. Specifically, based on the collision location's position in the world coordinate system, candidate points near the collision location are determined from the point cloud data contained in the point cloud map. The spatial distance between the collision location and each candidate point is calculated. Optionally, the candidate point with the shortest spatial distance can be selected as the target point corresponding to the collision location.

[0120] Secondly, based on the collision location and the target point, a target obstacle area including the collision location is determined.

[0121] To include the collision location within the obstacle region of the point cloud map, the spatial area near the line connecting the collision location and the target point can be considered as the newly added obstacle region. Optionally, a spherical bounding box can be constructed with the collision location as the center and the line connecting the collision location and the target point as the radius; the area within this bounding box is the newly added obstacle region. Alternatively, based on the determined target point, obstacles corresponding to the target point can be identified in the point cloud map as target obstacles. The location of the target point contained within the target obstacle can be used as the starting position, and the collision location as the ending position. The point cloud data contained within the target obstacle can be translated from the starting position to the ending position, and the area containing the translated point cloud data contained within the target obstacle can be considered as the target obstacle region.

[0122] Then, the obstacle region is updated based on the target obstacle region.

[0123] Finally, based on the updated obstacle area, the motion trajectory of the robotic arm is replanned so that the robotic arm moves according to the replanned motion trajectory.

[0124] Furthermore, the newly added obstacle area determined by the collision location and target point, combined with the original obstacle area pre-marked in the point cloud map, constitutes the updated obstacle area. This updated obstacle area includes the locations of obstacles not anticipated during the operation, as well as the locations of obstacles collected beforehand.

[0125] As can be seen, even if the position of the obstacle shifts during the operation, the obstacle avoidance method provided in this manual can update the position of the shifted obstacle to the obstacle area in the point cloud map, making the replanned motion trajectory more accurate, avoiding the obstacle area where the displacement changes, and eliminating the need for a second scan during the operation using acquisition equipment, thus simplifying the operation process.

[0126] The above describes one or more obstacle avoidance methods provided in this specification. Based on the same concept, this specification also provides corresponding obstacle avoidance devices, such as... Figure 5 As shown.

[0127] Figure 5 This specification provides a schematic diagram of an obstacle avoidance device, which specifically includes:

[0128] The obstacle area determination module 300 is used to construct a point cloud map based on the point cloud data of each obstacle sent by the acquisition device, and to mark the obstacle area in the point cloud map;

[0129] Planning module 302 is used to plan the motion trajectory of the robotic arm based on the obstacle area, so that the robotic arm moves according to the motion trajectory;

[0130] The judgment module 304 is used to monitor the sensor data sent by the collision sensor configured on the robotic arm, and determine whether the robotic arm has collided with an obstacle based on the sensor data.

[0131] The collision position determination module 306 is used to determine the collision position of the robotic arm and the obstacle when it is determined that the robotic arm has collided with the obstacle.

[0132] The update module 308 is used to update the obstacle area according to the collision location, and replan the motion trajectory of the robotic arm according to the updated obstacle area, so that the robotic arm moves according to the replanned motion trajectory.

[0133] Optionally, the judgment module 304 is specifically used to determine whether the sensor data is greater than a preset safety threshold; if not, not to send a control command to make the robotic arm continue to move along the motion trajectory; if yes, to send a control command to the robotic arm to make the robotic arm move away from the obstacle.

[0134] Optionally, the collision position determination module 306 is specifically configured to: acquire the receiving time of the sensor data; determine the position of the robotic arm at the receiving time based on the motion trajectory, as the desired position of the robotic arm; determine the offset vector of the robotic arm on the motion trajectory based on the sensor data; and determine the collision position where the robotic arm collides with the obstacle based on the desired position of the robotic arm on the motion trajectory and the offset vector of the robotic arm on the motion trajectory.

[0135] Optionally, the collision position determination module 306 is specifically used to: obtain the impedance stiffness corresponding to each preset direction; determine whether the direction of the sensor data coincides with each preset direction; if so, determine the offset vector of the robotic arm on the motion trajectory based on the impedance stiffness corresponding to the coincident preset direction and the sensor data; if not, determine the offset vector of the robotic arm on the motion trajectory based on each preset direction and the sensor data.

[0136] Optionally, the collision position determination module 306 is specifically configured to: determine the data components of the sensor data in each preset direction based on the sensor data; for each preset direction, determine the offset vector of the robotic arm in that preset direction based on the predetermined impedance stiffness corresponding to that preset direction and the data components in that preset direction; and determine the offset vector of the robotic arm on the motion trajectory based on the offset vector of the robotic arm in each preset direction.

[0137] Optionally, the update module 308 is specifically configured to: determine the distance between the collision location and each point cloud data contained in the point cloud map based on the collision location; determine a target point from each point cloud data based on the determined distance; determine a target obstacle area containing the collision location based on the collision location and the target point; and update the obstacle area based on the target obstacle area.

[0138] Optionally, the update module 308 is specifically used to construct a spherical region, with the collision location as the center of the sphere and the distance between the collision location and the target point as the radius of the sphere, as the target obstacle region containing the collision location.

[0139] Optionally, the collision sensor is configured at the end of the robotic arm and the joint mechanism;

[0140] Optionally, the determination module 304 is specifically used to determine that the end of the robotic arm collides with the obstacle when the sensor data sent by the collision sensor configured on the end of the robotic arm is greater than a preset safety threshold; and / or to determine that the joint mechanism of the robotic arm collides with the obstacle when the sensor data sent by the collision sensor configured on the joint structure of the robotic arm is greater than a preset safety threshold.

[0141] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The obstacle avoidance methods provided.

[0142] This instruction manual also provides Figure 6 The diagram shows a schematic structural representation of the electronic device. Figure 6 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The obstacle avoidance method described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0143] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0144] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0145] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0146] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0147] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0148] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0149] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0150] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0151] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0152] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0153] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0154] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0155] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0156] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0157] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0158] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. An obstacle avoidance method, characterized in that, The method includes: Based on the point cloud data of each obstacle sent by the acquisition device, a point cloud map is constructed, and the obstacle areas are marked in the point cloud map; Based on the obstacle area, plan the movement trajectory of the robotic arm, and make the robotic arm move according to the movement trajectory; Monitor the sensor data sent by the collision sensor configured on the robotic arm, and determine whether the robotic arm has collided with an obstacle based on the sensor data; When it is determined that the robotic arm collides with the obstacle, the collision position of the robotic arm and the obstacle is determined, specifically including: acquiring the receiving time of the sensor data; determining the position of the robotic arm at the receiving time according to the motion trajectory, as the expected position of the robotic arm; and acquiring the impedance stiffness corresponding to each preset direction. When the direction of the sensor data does not coincide with the preset directions: based on the sensor data, determine the data components of the sensor data in each preset direction; for each preset direction, based on the predetermined impedance stiffness corresponding to the preset direction and the data components in the preset direction, determine the offset vector of the robotic arm in that preset direction; based on the offset vector of the robotic arm in each preset direction, determine the offset vector of the robotic arm on the motion trajectory; based on the desired position of the robotic arm and the offset vector of the robotic arm on the motion trajectory, determine the collision position where the robotic arm collides with the obstacle. The obstacle area is updated based on the collision location, and the movement trajectory of the robotic arm is replanned based on the updated obstacle area, so that the robotic arm moves according to the replanned movement trajectory.

2. The method as described in claim 1, characterized in that, Based on the sensor data, determining whether the robotic arm has collided with an obstacle specifically includes: Determine whether the sensor data is greater than a preset safety threshold; If not, no control command is sent, and the robotic arm continues to move along the motion trajectory; If so, send a control command to the robotic arm to move it away from the obstacle.

3. The method as described in claim 1, characterized in that, When the direction of the sensor data coincides with each of the preset directions, the offset vector of the robotic arm on the motion trajectory is determined based on the impedance stiffness corresponding to the coincident preset direction and the sensor data.

4. The method as described in claim 1, characterized in that, The obstacle region is updated based on the determined collision location, specifically including: Based on the collision location, determine the distance between the collision location and each point cloud data contained in the point cloud map, and determine the target point from each point cloud data based on the determined distance; Based on the collision location and the target point, determine the target obstacle area that includes the collision location; Update the obstacle region based on the target obstacle region.

5. The method as described in claim 4, characterized in that, Based on the collision location and the target point, a target obstacle area including the collision location is determined, specifically including: Using the collision location as the center of the sphere and the distance between the collision location and the target point as the radius of the sphere, a spherical region is constructed as the target obstacle region containing the collision location.

6. The method as described in claim 1, characterized in that, The collision sensor is configured at the end of the robotic arm and at the joint mechanism; Monitoring the sensor data sent by the collision sensors configured on the robotic arm, and determining whether the robotic arm has collided with an obstacle based on the sensor data, specifically includes: When the sensor data sent by the collision sensor configured on the end effector of the robotic arm exceeds a preset safety threshold, it is determined that the end effector of the robotic arm has collided with the obstacle; and / or When the sensor data sent by the collision sensor configured on the joint mechanism of the robotic arm is greater than a preset safety threshold, it is determined that the joint mechanism of the robotic arm has collided with the obstacle.

7. An obstacle avoidance device, characterized in that, include: The obstacle area determination module is used to construct a point cloud map based on the point cloud data of each obstacle sent by the acquisition device, and to mark the obstacle area in the point cloud map; The planning module is used to plan the motion trajectory of the robotic arm based on the obstacle area, so that the robotic arm moves according to the motion trajectory; The judgment module is used to monitor the sensor data sent by the collision sensor configured on the robotic arm, and determine whether the robotic arm has collided with an obstacle based on the sensor data. The collision position determination module is used to determine the collision position of the robotic arm and the obstacle when it is determined that the robotic arm collides with the obstacle. Specifically, it includes: acquiring the receiving time of the sensor data; determining the position of the robotic arm at the receiving time according to the motion trajectory as the expected position of the robotic arm; and acquiring the impedance stiffness corresponding to each preset direction. When the direction of the sensor data does not coincide with the preset directions: based on the sensor data, determine the data components of the sensor data in each preset direction; for each preset direction, based on the predetermined impedance stiffness corresponding to the preset direction and the data components in the preset direction, determine the offset vector of the robotic arm in that preset direction; based on the offset vector of the robotic arm in each preset direction, determine the offset vector of the robotic arm on the motion trajectory; based on the desired position of the robotic arm and the offset vector of the robotic arm on the motion trajectory, determine the collision position where the robotic arm collides with the obstacle. An update module is used to update the obstacle area based on the collision location, and to replan the motion trajectory of the robotic arm based on the updated obstacle area, so that the robotic arm moves according to the replanned motion trajectory.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 6.

Images