A multifunctional industrial robot mechanical arm path obstacle avoidance optimization method and device

Abstract

The application is suitable for the technical field of image processing, and provides a multifunctional industrial robot mechanical arm path obstacle avoidance optimization method and device.The multifunctional industrial robot mechanical arm path obstacle avoidance optimization method comprises the following steps: collecting a current depth image of an operation table area through a structured light depth camera; extracting an obstacle area in the current depth image according to respective pixel values and depth information of the current depth image and a standard image; constructing a redundant three-dimensional area according to the obstacle area; when a plurality of track points in an original control path are located in the redundant three-dimensional area, extracting a projection position of a cross track point on a surface of the redundant three-dimensional area; replacing the cross track point in the original control path with the projection position to obtain a target control path, and controlling a mechanical arm to avoid obstacles based on the target control path.The above scheme ensures the accuracy and safety of robot path planning through the innovative application of the redundant three-dimensional area and the cross track point replacement mechanism.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image processing, and particularly relates to a multifunctional industrial robot mechanical arm path obstacle avoidance optimization method and device. BACKGROUND

[0002] With the continuous development of industrial automation, robot technology is increasingly widely used in manufacturing, especially the application of multifunctional industrial robot mechanical arms, which has become the key to improving production efficiency and precision. Path planning of mechanical arms is one of the important problems in robot control, which directly affects the working efficiency and safety of mechanical arms in complex working environments. In order to solve the path conflict problem caused by obstacles during the operation of industrial robots, path obstacle avoidance technology has been widely concerned.

[0003] Existing path planning and obstacle avoidance technology is mainly based on two ways: one is static path planning, which calculates the optimal motion trajectory of the mechanical arm by analyzing the position of the static obstacle in the environment; the second is dynamic path planning, which considers the dynamic changes of obstacles and adjusts the path of the mechanical arm in real time. However, these methods have certain limitations, especially in the case of complex and rapidly changing obstacles in the environment, the existing technology is difficult to adjust the path in time and accurately, resulting in low path planning efficiency, and even may cause damage to the mechanical arm and surrounding equipment. SUMMARY

[0004] Therefore, the embodiments of the present application provide a multifunctional industrial robot mechanical arm path obstacle avoidance optimization method and device to solve the technical problem that the existing technology is difficult to adjust the path in time and accurately.

[0005] The first aspect of the embodiments of the present application provides a multifunctional industrial robot mechanical arm path obstacle avoidance optimization method, which comprises:

[0006] Obtain the original control path of the mechanical arm and the standard image, and collect the current depth image of the operation table area through the structured light depth camera; the standard image refers to the image collected when there is no obstacle in the operation table area;

[0007] According to the pixel value and depth information corresponding to the current depth image and the standard image respectively, the obstacle region in the current depth image is extracted;

[0008] According to the obstacle region, a redundant three-dimensional region is constructed; the redundant space region includes an obstacle three-dimensional region and a redundant three-dimensional region;

[0009] extract a projection position of the intersection trajectory point on the surface of the redundant three-dimensional region when the trajectory points in the original control path are in the redundant three-dimensional region; the intersection trajectory point refers to the trajectory point in the redundant three-dimensional region;

[0010] replace the intersection trajectory point in the original control path with the projection position to obtain a target control path, and control the robot arm to avoid obstacles based on the target control path.

[0011] Further, the step of extracting the obstacle region in the current depth image according to the pixel value and depth information corresponding to the current depth image and the standard image respectively comprises:

[0012] calculating a pixel difference value between the same pixel positions of the current depth image and the standard image;

[0013] regarding the pixel position with a pixel difference value greater than a first threshold value as an abnormal pixel position;

[0014] extracting an abnormal pixel region composed of continuous abnormal pixel positions;

[0015] extracting first depth information of the abnormal pixel region in the current depth image and second depth information of the abnormal pixel region in the standard image;

[0016] if the difference between the first depth information and the second depth information exceeds a second threshold value, regarding the abnormal pixel region as the obstacle region.

[0017] Further, the step of constructing the redundant three-dimensional region according to the obstacle region comprises:

[0018] extending the obstacle region based on a preset extension coefficient to obtain a current image region;

[0019] converting the current image region into X-axis coordinates and Y-axis coordinates in the actual environment;

[0020] extracting a minimum depth value in the first depth information corresponding to the obstacle region;

[0021] converting the minimum depth value into Z-axis coordinates;

[0022] regarding a plane composed of the X-axis coordinates corresponding to the current image region, the Y-axis coordinates corresponding to the current image region, and the Z-axis coordinates as a three-dimensional region top surface;

[0023] extending the three-dimensional region top surface in a direction perpendicular to the Z-axis direction to the surface of the operation platform in the direction of the operation platform to obtain the redundant three-dimensional region.

[0024] Further, the step of extracting the projection position of the intersection trajectory points on the surface of the redundant three-dimensional region when the plurality of trajectory points in the original control path are in the redundant three-dimensional region comprises:

[0025] When all the intersection trajectory points are on the surface of the three-dimensional region, the plurality of intersection trajectory points are taken as the projection position;

[0026] When not all the intersection trajectory points are on the surface of the three-dimensional region, according to the number characteristics of the plurality of intersection trajectory points, the projection position of the intersection trajectory points on the surface of the redundant three-dimensional region is extracted.

[0027] Further, the step of extracting the projection position of the intersection trajectory points on the surface of the redundant three-dimensional region when not all the intersection trajectory points are on the surface of the three-dimensional region, according to the number characteristics of the plurality of intersection trajectory points, comprises:

[0028] Extracting a surface intersection trajectory point in the plurality of intersection trajectory points; the surface intersection trajectory point refers to an intersection trajectory point on the surface of the redundant three-dimensional region;

[0029] If the number of the surface intersection trajectory points is two, a first line segment corresponding to the two surface intersection trajectory points is extracted;

[0030] Obtaining a plurality of edge points on the contour of the top surface of the three-dimensional region;

[0031] Respectively constructing a current triangle based on the edge point and the first line segment; wherein the edge point is a vertex of the current triangle, and the first line segment is a bottom side of the current triangle;

[0032] Extracting a minimum area current triangle from the plurality of current triangles;

[0033] Taking two opposite sides of the minimum area current triangle as the projection position of the surface of the three-dimensional region; the two opposite sides refer to two sides of the current triangle other than the bottom side;

[0034] If the number of the surface intersection trajectory points exceeds two, a plurality of surface intersection trajectory points are sorted according to the original trajectory point order;

[0035] Extracting a second line segment formed by a first surface intersection trajectory point and a last surface intersection trajectory point; the first surface intersection trajectory point refers to an intersection trajectory point ranked first in the plurality of surface intersection trajectory points, and the last surface intersection trajectory point refers to an intersection trajectory point ranked last in the plurality of surface intersection trajectory points;

[0036] Respectively constructing a target triangle based on the edge point and the second line segment; wherein the edge point is a vertex of the target triangle, and the second line segment is a bottom side of the target triangle;

[0037] extracting a minimum area target triangle from the plurality of target triangles;

[0038] taking two opposite sides of the minimum area target triangle as the projection position of the three-dimensional region surface.

[0039] Further, after the step of constructing a redundant three-dimensional region according to the obstacle region, further comprising:

[0040] If the number of intersection track points is single, controlling the robot arm to avoid obstacles based on the original control path.

[0041] Further, after the steps of extracting first depth information of an abnormal pixel region in the current depth image and extracting second depth information of a difference pixel region in the standard image, further comprising:

[0042] If the difference between the first depth information and the second depth information does not exceed a second threshold value, controlling the robot arm to perform a specified action based on the original control path;

[0043] In the process of performing the specified action, collecting a real-time depth image of the operation table region by a structured light depth camera;

[0044] According to the pixel value and depth information corresponding to the real-time depth image and the standard image respectively, extracting an obstacle region in the real-time depth image;

[0045] If no obstacle is identified, continuing to control the robot arm to perform the specified action based on the original control path;

[0046] If an obstacle is identified, performing an obstacle avoidance process.

[0047] A second aspect of the embodiment of the application provides a multifunctional industrial robot arm path obstacle avoidance optimization device, comprising:

[0048] An acquisition unit is configured to acquire an original control path of a robot arm and a standard image, and collect a current depth image of an operation table region by a structured light depth camera; the standard image is an image collected when there is no obstacle in the operation table region;

[0049] A first extraction unit is configured to extract an obstacle region in the current depth image according to the pixel value and depth information corresponding to the current depth image and the standard image respectively;

[0050] A construction unit is configured to construct a redundant three-dimensional region according to the obstacle region; the redundant space region comprises an obstacle three-dimensional region and a redundant three-dimensional region.

[0051] The second extraction unit is configured to extract a projection position of a cross trajectory point on a surface of the redundant three-dimensional region when the plurality of trajectory points in the original control path are in the redundant three-dimensional region; the cross trajectory point refers to a trajectory point in the redundant three-dimensional region.

[0052] The obstacle avoidance unit is configured to replace the cross trajectory point in the original control path with the projection position to obtain a target control path, and control the robot arm to avoid obstacles based on the target control path.

[0053] The third aspect of the embodiment of the present application provides a terminal device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the steps in the multi-functional industrial robot arm path obstacle avoidance optimization method of the first aspect.

[0054] The fourth aspect of the embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executable on a processor to implement the steps in the multi-functional industrial robot arm path obstacle avoidance optimization method of the first aspect.

[0055] Compared with the prior art, the beneficial effects of the embodiment of the present application are that the current depth image of the operation table area is obtained through the structured light depth camera, and the image data of the unobstructed environment in the standard image can be combined to detect the position and shape of the obstacle in the operation area in real time. Unlike traditional static or simple depth perception-based path planning methods, the present application dynamically adjusts the path through real-time captured depth information, effectively avoiding collision problems caused by untimely or inaccurate path planning. This dynamic obstacle avoidance mechanism greatly improves the flexibility and adaptability of the robot in complex environments. The present application introduces the concept of redundant three-dimensional region, which combines the three-dimensional region where the obstacle is located with the redundant region. The redundant three-dimensional region not only considers the influence of the obstacle itself, but also further optimizes the path planning by reserving a certain space range. The trajectory points of the robot arm in the redundant three-dimensional region are effectively replaced by their projection positions on the surface of the redundant region, thereby avoiding conflicts between the trajectory and the obstacle and improving the accuracy and safety of path planning. In the original control path, when multiple trajectory points are in the redundant three-dimensional region, the present application extracts the projection positions of the cross trajectory points on the surface of the redundant three-dimensional region and replaces them, avoiding interference between the path and the obstacle during the operation of the robot arm. Through this technical means, path planning not only ensures a safe distance between the robot arm and the obstacle, but also improves the efficiency of the robot arm when performing tasks, reducing multiple iteration calculations and unnecessary path adjustments, thereby reducing the computational burden of the system. The present application can obtain real-time depth information of the current environment and dynamically adjust the path according to environmental changes, so that the robot arm can quickly adapt to different working environments. Especially in the face of complex and variable operation table areas, accurate path adjustment can reduce the number of robot arm pauses and re-planning, improving the efficiency of industrial robots in actual production. In addition, the application of path obstacle avoidance optimization method also improves the adaptability of the robot in unknown environments and reduces work interruptions caused by environmental changes. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or related technical descriptions. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0057] Figure 1 A schematic flow chart of a multifunctional industrial robot arm path obstacle avoidance optimization method provided by the present application is shown;

[0058] Figure 2A schematic diagram of a multifunctional industrial robot mechanical arm path obstacle avoidance optimization device is shown.

[0059] Figure 3 A schematic diagram of a terminal device is shown. DETAILED DESCRIPTION

[0060] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, and circuits are omitted so as not to obscure the description of the present application with unnecessary detail.

[0061] The embodiments of the present application provide a multifunctional industrial robot mechanical arm path obstacle avoidance optimization method and device to solve the technical problem of not being able to adjust the path in time and accurately in the prior art.

[0062] First, the present application provides a multifunctional industrial robot mechanical arm path obstacle avoidance optimization method. Please refer to Figure 1 , Figure 1 A schematic flow chart of a multifunctional industrial robot mechanical arm path obstacle avoidance optimization method is shown. As Figure 1 shown, the multifunctional industrial robot mechanical arm path obstacle avoidance optimization method can include the following steps:

[0063] Step 101: Obtain the original control path of the mechanical arm and the standard image, and collect the current depth image of the operation table area through the structured light depth camera; the standard image refers to the image collected when there is no obstacle in the operation table area;

[0064] The original control path is the initial motion trajectory of the mechanical arm when performing a task, which is a path planned in advance according to the task requirements.

[0065] The standard image is an image of the operation table area collected without any obstacles. It serves as a reference for subsequent comparison and analysis.

[0066] The current depth image refers to the image collected in real time through the structured light depth camera, which can obtain the depth information of the operation table area, showing the relative distance between each object in the current operation environment and the camera, and helping to determine the existence and position of the obstacle.

[0067] Step 102: According to the pixel value and depth information corresponding to the current depth image and the standard image respectively, extract the obstacle region in the current depth image;

[0068] Since there is limitation in identifying the obstacle region only by a single pixel value or depth value, the application identifies the obstacle region in combination with pixel value difference and depth information, and the specific logic is as follows:

[0069] Specifically, the step 102 specifically includes steps 1021 to 1025:

[0070] Step 1021: Calculate the pixel difference value between the same pixel position between the current depth image and the standard image;

[0071] Step 1022: The pixel position with a pixel difference value greater than a first threshold value is regarded as an abnormal pixel position;

[0072] When the pixel difference value is greater than the first threshold value, it indicates that the pixel information at this position changes, which needs to be determined as an abnormal pixel position.

[0073] Step 1023: Extract an abnormal pixel region composed of continuous abnormal pixel positions;

[0074] By aggregating these abnormal pixel points (i.e. pixel points with a depth difference value greater than the first threshold value) by position, an "abnormal pixel region" is formed.

[0075] Step 1024: Extract the first depth information of the abnormal pixel region in the current depth image, and extract the second depth information of the difference pixel region in the standard image;

[0076] This step is to compare the depth information of the extracted abnormal pixel region in more detail. For each abnormal pixel region, the depth information corresponding to the abnormal pixel region in the current depth image is extracted (i.e. the first depth information). At the same time, the depth information of the corresponding region in the standard image is extracted (i.e. the second depth information). The two depth information are used to further determine whether these regions are truly obstacles.

[0077] Step 1025: If the difference between the first depth information and the second depth information exceeds a second threshold value, the difference pixel region is regarded as an obstacle region.

[0078] This step is to verify the depth difference. By comparing the difference between the first depth information extracted in the current image and the second depth information in the standard image, if the difference is greater than the second threshold value, it indicates that there is a significant depth difference between the region in the current depth image and the standard image, which may be caused by an obstacle. Therefore, the region will be marked as an obstacle region.

[0079] The second threshold is set to ensure that only significant depth changes are considered as obstacles. If the difference is less than the threshold, it is likely that the depth change is caused by noise or other non-obstacle factors, and these areas are not considered as obstacles.

[0080] In the corresponding embodiments of steps 1021 to 1025, first, the pixel value difference of each pixel point is calculated, then the threshold is used to determine which positions have abnormal differences, and finally these abnormal positions are aggregated into a region. Finally, by comparing the depth information of the region with the depth information in the standard image, it is confirmed whether there is an obstacle. If the depth difference exceeds the set threshold, the region is considered as an obstacle region. This method can effectively extract the position of the obstacle from the real-time image and provide data support for path optimization. Through the two dimensions of pixel value difference and depth information, the abnormal region in the image is detected together.

[0081] As an optional embodiment of the present application, after step 1024, steps B1 to B5 are further included:

[0082] Step B1: if the difference between the first depth information and the second depth information does not exceed the second threshold, the original control path is used to control the robot arm to perform a specified action;

[0083] Since in the case of abnormal pixel difference and no abnormal depth information, it is impossible to determine whether there is an obstacle. During the movement of the robot arm, multiple real-time depth images in different directions can be collected, which can further determine whether there is an obstacle, so it is necessary to control the robot arm to perform a specified action based on the original control path. Real-time depth images are collected during the execution of the specified action for further detection and judgment.

[0084] Step B2: during the execution of the specified action, a real-time depth image of the operation table region is collected by a structured light depth camera;

[0085] The structured light depth camera will continuously monitor the operation table region and collect real-time depth images. This process ensures that three-dimensional data of the operation table can be obtained in real time during the execution of the robot arm task. The real-time image reflects the current environmental conditions of the operation table region, including potential obstacles or other changes.

[0086] Step B3: according to the pixel values and depth information corresponding to the real-time depth image and the standard image respectively, an obstacle region in the real-time depth image is extracted;

[0087] Step B3 is similar to step 102, which will not be repeated here.

[0088] Step B4: if no obstacle is identified, the original control path is used to control the robot arm to perform a specified action;

[0089] Step B5: If an obstacle is identified, an obstacle avoidance process is performed.

[0090] The obstacle avoidance process refers to the process corresponding to steps 103 to 105.

[0091] In the embodiment corresponding to steps B1 to B5, a structured light depth camera is used to monitor the depth information of the operation table area in real time, and by comparing with the standard image, obstacles are quickly identified. If there is no obstacle, the robot arm will continue to perform the task according to the original control path; if an obstacle is identified, the system will start the obstacle avoidance process to ensure that the robot arm can bypass the obstacle and complete the task. In this way, the robot arm can operate efficiently and intelligently in a dynamic environment while avoiding collisions or task failures caused by obstacles.

[0092] Step 103: Construct a redundant three-dimensional region according to the obstacle region; the redundant space region includes the obstacle three-dimensional region and the redundant three-dimensional region;

[0093] In order to ensure the safety of the robot arm path, a redundant region needs to be constructed on the basis of the obstacle. The redundant region is a safety buffer zone around the obstacle region, which is used to prevent the robot arm from colliding with the obstacle. The design of the redundant region ensures that the robot arm can avoid direct contact with the obstacle when adjusting the path. The construction logic of the redundant three-dimensional region is as follows:

[0094] Specifically, step 103 specifically includes steps 1031 to 1036:

[0095] Step 1031: Based on a preset extension coefficient, the obstacle region is extended to obtain a current image region;

[0096] The preset extension coefficient is a parameter set in advance, which is used to extend the obstacle region outward by a fixed distance. By setting the extension coefficient, the size of the redundant region can be adjusted as needed. For example, the larger the extension coefficient, the farther the boundary of the redundant region, ensuring that the robot arm has more safety space when avoiding obstacles.

[0097] The extension processing is to enlarge the obstacle region to ensure that the space around the obstacle is covered. The purpose of this step is to construct a larger redundant region than the actual region of the obstacle to avoid the robot arm approaching the obstacle when adjusting the path.

[0098] Step 1032: Convert the current image region to X-axis coordinates and Y-axis coordinates in the actual environment;

[0099] The conversion between the image coordinate system (pixel coordinates) and the physical coordinates in the actual environment (such as X-axis and Y-axis) is needed. The purpose of this step is to map the position in the image to the actual working space, which is achieved through the camera's internal parameters. This can make the position on the image correspond to the position on the actual operation platform.

[0100] Step 1033: Extract the minimum depth value in the first depth information corresponding to the obstacle region;

[0101] The first depth information is extracted from the current depth image, representing the distance of each pixel point from the camera. In the obstacle region, the nearest point in this region (i.e., the minimum depth value) needs to be found, which represents the nearest boundary of the obstacle. The minimum depth value is a key parameter for calculating the top of the redundant region. By extracting the minimum depth value, it avoids the collision between the robot arm and the obstacle in the subsequent movement process.

[0102] Step 1034: Convert the minimum depth value to Z-axis coordinates;

[0103] By converting the minimum depth value to Z-axis coordinates, the vertical position of the obstacle region in three-dimensional space is obtained. The depth value is actually the position on the Z-axis, indicating the distance of the obstacle from the camera, and the minimum depth value represents the nearest point of the obstacle.

[0104] Step 1035: Take the plane formed by the X-axis coordinates corresponding to the current image region, the Y-axis coordinates corresponding to the current image region, and the Z-axis coordinates as the top surface of the three-dimensional region;

[0105] Through the coordinate conversion and depth information extraction in the previous steps, the position of the obstacle region on the X, Y, and Z axes is obtained. Combining these points forms a plane, which serves as the top surface of the three-dimensional region. This top surface describes the upper boundary of the redundant three-dimensional region, ensuring that the upper part of the redundant region covers the obstacle region.

[0106] Step 1036: Extend the three-dimensional region top surface to the operation platform surface in the direction perpendicular to the Z-axis direction towards the operation platform to obtain the redundant three-dimensional region.

[0107] Starting from the three-dimensional region top surface, extend downward (perpendicular to the Z-axis direction) until reaching the surface of the operation platform. The purpose of this step is to ensure that the redundant three-dimensional region can cover the space below the obstacle region, thereby leaving enough obstacle avoidance space for the robot arm. This process ensures that the redundant region can effectively avoid collision between the robot arm and the obstacle, even if the height of the obstacle in the Z-axis direction changes greatly, the redundant region can effectively cover enough space.

[0108] In the embodiments corresponding to steps 1031 to 1036, a redundant space is created by extending the obstacle region and converting it into a three-dimensional region in the actual environment to ensure a safe path for the robot arm. In actual operation, a safety buffer zone is constructed around the obstacle by presetting an extension coefficient to avoid collision between the robot arm and the obstacle. This redundant three-dimensional region provides sufficient space for path adjustment to ensure that the robot arm can smoothly avoid the obstacle and continue to perform the task.

[0109] Step 104: Extract the projection position of the intersection trajectory points on the surface of the redundant three-dimensional region when the trajectory points in the original control path are in the redundant three-dimensional region; the intersection trajectory points refer to the trajectory points in the redundant three-dimensional region;

[0110] The key of this step is to check whether the trajectory points on the original control path of the robot arm enter the redundant three-dimensional region. If a certain trajectory point in the path enters the redundant region, this trajectory point needs to be adjusted. The adjustment method is to project these trajectory points on the surface of the redundant region to a new position, that is, to find a safe point that does not conflict with the obstacle.

[0111] Specifically, step 104 specifically includes steps 1041 to 1042:

[0112] Step 1041: When the plurality of intersection trajectory points are all on the surface of the three-dimensional region, the plurality of intersection trajectory points are taken as the projection positions;

[0113] When the plurality of intersection trajectory points are all on the surface of the three-dimensional region, it means that all the intersection trajectory points are located on the surface of the redundant three-dimensional region and do not need additional processing. In other words, all the trajectory points coincide with the surface of the redundant three-dimensional region, so these points can be taken as the projection positions themselves.

[0114] Since all the intersection trajectory points are already on the surface of the redundant three-dimensional region, the current positions of these trajectory points are directly taken as the projection positions. This is because the physical positions of these trajectory points have already coincided with the surface of the redundant region, so their projection positions are themselves.

[0115] Step 1042: When the plurality of intersection trajectory points are not all on the surface of the three-dimensional region, the projection positions of the intersection trajectory points on the surface of the redundant three-dimensional region are extracted according to the quantity characteristics of the plurality of intersection trajectory points.

[0116] When the trajectory points are not completely on the surface of the redundant three-dimensional region, the projection position refers to the process of mapping these trajectory points according to their actual positions to the surface of the redundant region. This mapping is completed through geometric transformation. The specific process is as follows:

[0117] In the embodiments corresponding to steps 1041 to 1042, the purpose is to ensure that all intersection trajectory points are correctly mapped to the surface of the redundant three-dimensional region to avoid collision between the mechanical arm path and the obstacle. By converting the actual positions of the intersection trajectory points into projected positions on the surface of the redundant region, the original control path can be adjusted to ensure that the mechanical arm travels along a safe path. If all intersection trajectory points are on the surface, these points are directly used as the projected positions; if some intersection trajectory points are not on the surface, they need to be mapped to the surface of the redundant region according to their number characteristics using geometric transformation.

[0118] Specifically, step 1042 specifically includes steps A1 to A11:

[0119] Step A1: Extract surface intersection trajectory points from the plurality of intersection trajectory points; the surface intersection trajectory points refer to intersection trajectory points on the surface of the redundant three-dimensional region;

[0120] Step A2: If the number of surface intersection trajectory points is two, extract a first line segment corresponding to the two surface intersection trajectory points;

[0121] When two surface intersection trajectory points of the original path are located on the surface of the redundant three-dimensional region, these two points define a line segment, referred to as a "first line segment". The two endpoints of this line segment are the two surface intersection trajectory points, respectively. It represents the part intersecting with the original path on the surface of the redundant three-dimensional region. This line segment is the basis for subsequent construction of triangles and calculation of projected positions.

[0122] Step A3: Obtain a plurality of edge points on the contour of the top surface of the three-dimensional region;

[0123] The top surface of the redundant three-dimensional region is its upper boundary, which can be a polygon or other geometric shape. The edge points are a plurality of sampling points (sampling frequency is set based on calculation accuracy) of the contour of the top surface of the redundant three-dimensional region. The edge points are used to construct triangles and further calculate the projected positions of the surface intersection trajectory points.

[0124] Step A4: Construct a current triangle based on the edge points and the first line segment, respectively; wherein the edge points are vertices of the current triangle, and the first line segment is a base of the current triangle;

[0125] By combining the edge points on the top surface of the redundant region with the first line segment, a triangle can be constructed. The edge points serve as the vertices of the triangle, and the first line segment serves as the base.

[0126] Step A5: Extract a minimum area current triangle from the plurality of current triangles;

[0127] From the plurality of constructed triangles, the triangle with the smallest area is selected.

[0128] Since the embodiment takes two opposite sides of the current triangle as the obstacle avoidance path, the minimum perimeter needs to be selected. The minimum perimeter corresponds to the minimum area triangle, so the minimum area current triangle needs to be extracted.

[0129] Step A6: taking two opposite sides of the minimum area current triangle as the projection position of the three-dimensional region surface; the two opposite sides refer to two sides of the current triangle other than the base side;

[0130] The current triangle has three sides, two of which are opposite the base side and are called opposite sides. The projection position is the path of the two opposite sides, indicating the final projection position of the intersection trajectory point on the redundant three-dimensional region surface.

[0131] Step A7: if the number of surface intersection trajectory points exceeds two, sorting the multiple surface intersection trajectory points according to the original trajectory point order;

[0132] When the number of surface intersection trajectory points exceeds two, it means that the distribution of trajectory points is more complex. At this time, the order of the trajectory points in the original control path needs to be obtained, and the order corresponding to each of the multiple surface intersection trajectory points is obtained. The multiple surface intersection trajectory points are arranged in ascending order to obtain multiple ordered surface intersection trajectory points.

[0133] Step A8: extracting a second line segment composed of the first surface intersection trajectory point and the last surface intersection trajectory point; the first surface intersection trajectory point refers to the intersection trajectory point ranked first among the multiple surface intersection trajectory points, and the last intersection trajectory point refers to the intersection trajectory point ranked last among the multiple surface intersection trajectory points;

[0134] The first surface intersection trajectory point is the intersection trajectory point ranked first among the multiple surface intersection trajectory points, i.e., the starting trajectory point of the original control path entering the redundant three-dimensional region.

[0135] The last surface intersection trajectory point is the intersection trajectory point ranked last among the multiple surface intersection trajectory points, i.e., the ending trajectory point of the original control path leaving the redundant three-dimensional region.

[0136] It can be understood that the first and last surface intersection trajectory points are obtained to better connect the surface intersection trajectory points with the original control path and avoid trajectory point drift.

[0137] The two surface intersection trajectory points constitute a second line segment. This line segment will serve as a new base side to help build a new triangle.

[0138] Step A9: constructing a target triangle based on the edge point and the second line segment respectively; wherein the edge point is a vertex of the target triangle, and the second line segment is a base of the target triangle;

[0139] Similar to the foregoing steps, a new triangle is constructed by combining the edge point and the second line segment. The edge point remains as a vertex, while the second line segment serves as a base.

[0140] Step A10: extracting a minimum-area target triangle from the plurality of target triangles;

[0141] From the plurality of constructed triangles, the triangle with the smallest area is selected.

[0142] Since the embodiment takes two opposite sides of the target triangle as the obstacle avoidance path, it is necessary to select the minimum perimeter. The minimum-area target triangle corresponds to the minimum perimeter, so it is necessary to extract the minimum-area target triangle.

[0143] Step A11: taking two opposite sides of the minimum-area target triangle as the projection position of the three-dimensional region surface.

[0144] The target triangle has three sides, two of which are opposite to the base and are called opposite sides. The projection position is the path of the two opposite sides, indicating the final projection position of the intersection trajectory point on the redundant three-dimensional region surface.

[0145] In the embodiments corresponding to steps A1 to A11, the projection position of the surface intersection trajectory point is accurately extracted by constructing a plurality of triangles and selecting the smallest triangle according to the area. For a small number of surface intersection trajectory points (such as two), a triangle is constructed directly using the edge point and the first line segment; for a larger number of surface intersection trajectory points, a second line segment is generated by the first and last surface intersection trajectory points, and a target triangle is constructed. Finally, the projection position of the surface intersection trajectory point on the redundant three-dimensional region surface is determined by the opposite sides of the minimum-area triangle. This method ensures the accuracy of the projection position and effectively helps to adjust the motion path of the robot arm.

[0146] As an embodiment of the present application, after step 103, it further includes: if the number of intersection trajectory points is single, controlling the robot arm to avoid obstacles based on the original control path.

[0147] If the number of intersection trajectory points is single, it means that only one point intersects with the redundant region surface, or in other words, the original path only contacts the redundant region surface at a specific point. Therefore, there is no need to perform the obstacle avoidance process, and the original control path can be directly used to achieve obstacle avoidance.

[0148] Step 105: Replace the intersection trajectory points in the original control path with the projection positions to obtain a target control path, and control the robot arm based on the target control path to avoid obstacles.

[0149] In this step, the trajectory points entering the redundant three-dimensional region are replaced with new projection points to generate a new target control path. This target path ensures that the robot arm can avoid obstacles and safely execute tasks according to the new path.

[0150] In the embodiments corresponding to steps 101 to 105, the current depth image of the operation table region is obtained by the structured light depth camera, and the image data of the obstacle-free environment in the standard image is combined to detect the position and shape of the obstacle in the operation region in real time. Unlike traditional static or simple depth perception-based path planning methods, the present application dynamically adjusts the path based on real-time captured depth information, effectively avoiding collision problems caused by untimely or inaccurate path planning. This dynamic obstacle avoidance mechanism greatly improves the flexibility and adaptability of the robot in complex environments. The present application introduces the concept of redundant three-dimensional region, which combines the three-dimensional region where the obstacle is located with the redundant region. The redundant three-dimensional region not only considers the influence of the obstacle itself, but also further optimizes the path planning by reserving a certain space range. The trajectory points of the robot arm in the redundant three-dimensional region will be effectively replaced with their projection positions on the surface of the redundant region, thereby avoiding conflicts between the trajectory and the obstacle, improving the accuracy and safety of path planning. In the original control path, when multiple trajectory points are in the redundant three-dimensional region, the present application extracts the projection positions of the intersection trajectory points on the surface of the redundant three-dimensional region and replaces them, avoiding interference between the path and the obstacle during the operation of the robot arm. Through this technical means, path planning not only ensures a safe distance between the robot arm and the obstacle, but also improves the efficiency of the robot arm in executing tasks, reducing multiple iterative calculations and unnecessary path adjustments, thereby reducing the computational burden of the system. The present application can obtain real-time depth information of the current environment and dynamically adjust the path according to environmental changes, so that the robot arm can quickly adapt to different working environments. Especially in the face of complex and variable operation table regions, accurate path adjustment can reduce the number of robot arm pauses and re-planning, improving the efficiency of industrial robots in actual production. In addition, the application of path obstacle avoidance optimization method also improves the adaptability of the robot in unknown environments, reducing work interruptions caused by environmental changes.

[0151] As Figure 2 The present application provides a multifunctional industrial robot arm path obstacle avoidance optimization device, please see Figure 2 , Figure 2 shows a schematic diagram of a multifunctional industrial robot arm path obstacle avoidance optimization device provided by the present application,Figure 2 The multifunctional industrial robot mechanical arm path obstacle avoidance optimization device shown comprises:

[0152] An acquisition unit 21 is configured to acquire an original control path of a mechanical arm and a standard image, and collect a current depth image of an operating table area through a structured light depth camera; the standard image refers to an image collected when there is no obstacle in the operating table area;

[0153] A first extraction unit 22 is configured to extract an obstacle region in the current depth image according to respective pixel values and depth information of the current depth image and the standard image;

[0154] A construction unit 23 is configured to construct a redundant three-dimensional region according to the obstacle region; the redundant three-dimensional region comprises an obstacle three-dimensional region and a redundant three-dimensional region;

[0155] A second extraction unit 24 is configured to extract a projection position of a cross trajectory point on a surface of the redundant three-dimensional region when a plurality of trajectory points in the original control path are in the redundant three-dimensional region; the cross trajectory point refers to a trajectory point in the redundant three-dimensional region;

[0156] An obstacle avoidance unit 25 is configured to replace the cross trajectory point in the original control path with the projection position to obtain a target control path, and control the mechanical arm to avoid obstacles based on the target control path.

[0157] This invention provides a multifunctional obstacle avoidance optimization device for industrial robot arms. It acquires the current depth image of the operating platform area using a structured light depth camera and combines it with image data of the obstacle-free environment in a standard image to detect the position and shape of obstacles within the operating area in real time. Unlike traditional static or simple depth-sensing-based path planning methods, this invention dynamically adjusts the path using real-time captured depth information, effectively avoiding collisions caused by untimely or inaccurate path planning. This dynamic obstacle avoidance mechanism significantly improves the robot's flexibility and adaptability in complex environments. This invention introduces the concept of redundant three-dimensional regions, combining the three-dimensional region containing the obstacle with a redundant region. The redundant three-dimensional region not only considers the impact of the obstacle itself but also further optimizes path planning by reserving a certain spatial range. The trajectory points of the robot arm within the redundant three-dimensional region are effectively replaced by their projected positions on the surface of the redundant region, thereby avoiding conflicts between the trajectory and obstacles and improving the accuracy and safety of path planning. In the original control path, when multiple trajectory points are located in redundant 3D regions, this invention extracts and replaces the projection positions of intersecting trajectory points on the surface of the redundant 3D regions, thus avoiding interference between the robotic arm's path and obstacles during operation. Through this technique, path planning not only ensures a safe distance between the robotic arm and obstacles but also improves the efficiency of the robotic arm in performing tasks, reducing multiple iterative calculations and unnecessary path adjustments, thereby reducing the computational burden on the system. This invention can acquire real-time depth information of the current environment and dynamically adjust the path according to environmental changes, enabling the robotic arm to quickly adapt to different working environments. Especially when facing complex and variable operating platform areas, precise path adjustments can reduce the number of pauses and replanning attempts, improving the efficiency of industrial robots in actual production. Furthermore, the application of path obstacle avoidance optimization methods also improves the robot's adaptability to unknown environments, reducing work interruptions caused by environmental changes.

[0158] Figure 3 This is a schematic diagram of a terminal device provided in an embodiment of the present invention. Figure 3 As shown, a terminal device 3 in this embodiment includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a path obstacle avoidance optimization program for a multi-functional industrial robot arm. When the processor 30 executes the computer program 32, it implements the steps described in the embodiments of the multi-functional industrial robot arm path obstacle avoidance optimization method, for example... Figure 1 Steps 101 to 105 are shown. Alternatively, when the processor 30 executes the computer program 32, it implements the functions of each unit in the above-described device embodiments, for example... Figure 2 The function of the unit shown.

[0159] By way of example, the computer program 32 can be segmented into one or more units stored in the memory 31 and executed by the processor 30 to accomplish the present application. The one or more units can be a series of computer program instruction segments capable of accomplishing a specific function, which are used to describe the execution process of the computer program 32 in the terminal device 3. For example, the computer program 32 can be segmented into units with specific functions as follows:

[0160] An acquisition unit is configured to acquire an original control path of a robot arm and a standard image, and collect a current depth image of an operation table region by a structured light depth camera; the standard image refers to an image collected when there is no obstacle in the operation table region;

[0161] A first extraction unit is configured to extract an obstacle region in the current depth image according to pixel values and depth information corresponding to the current depth image and the standard image respectively;

[0162] A construction unit is configured to construct a redundant three-dimensional region according to the obstacle region; the redundant space region includes an obstacle three-dimensional region and a redundant three-dimensional region;

[0163] A second extraction unit is configured to extract a projection position of an intersection trajectory point on a surface of the redundant three-dimensional region when a plurality of trajectory points in the original control path are in the redundant three-dimensional region; the intersection trajectory point refers to a trajectory point in the redundant three-dimensional region;

[0164] An obstacle avoidance unit is configured to replace the intersection trajectory point in the original control path with the projection position to obtain a target control path, and control the robot arm to avoid obstacles based on the target control path.

[0165] The terminal device includes but is not limited to the processor 30 and the memory 31. Those skilled in the art can understand that, Figure 3 The terminal device 3 is only an example and does not constitute a limitation on the terminal device 3, and can include more or fewer components than the illustration, or combine certain components, or different components, for example, the terminal device can also include an input / output device, a network access device, a bus, etc.

[0166] The processor 30 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0167] The memory 31 can be an internal storage unit of the terminal device 3, for example, a hard disk or a memory of the terminal device 3. The memory 31 can also be an external storage device of the terminal device 3, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 31 can include both the internal storage unit and the external storage device of the terminal device 3. The memory 31 is used to store the computer program and other programs and data required by the roaming control device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0168] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0169] It should be noted that the information interaction, execution process, etc. between the above devices / units, since based on the same concept as the method embodiments of the present application, the specific functions and the technical effects brought by it can be referred to the method embodiments part, and will not be repeated here.

[0170] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the present application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0171] The embodiment of the present application further provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to realize the steps in each method embodiment.

[0172] The embodiment of the present application provides a computer program product, when the computer program product runs on a mobile terminal, so that the mobile terminal executes to realize the steps in each method embodiment.

[0173] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the present application realizes all or part of the processes in the above-mentioned embodiment methods, which can be completed by a computer program instructing related hardware. The computer program can be stored in a computer readable storage medium, and the computer program can realize the steps in each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, such as U disk, mobile hard disk, magnetic disk or optical disk, etc.

[0174] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0175] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0176] In the embodiments provided by the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic. The division of the modules or units is merely a logical function division. There can be another division manner in actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the display or discussion of the coupling or direct coupling or communication connection between the units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or in other forms.

[0177] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, which can be located in one place or distributed on a plurality of network units.

[0178] It should be understood that when used in the specification and the appended claims of the present application, the term "comprising" indicates the presence of described features, integers, steps, operations, elements, and / or components, but does not exclude one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0179] It should also be understood that the term "and / or" used in the specification and the appended claims of the present application means any combination of one or more of the associated listed items and all possible combinations thereof, and includes these combinations.

[0180] As used in the specification and the appended claims of the present application, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to monitoring" depending on the context. Similarly, the phrase "if it is determined" or "if it is monitored [that a described condition or event] can be interpreted depending on the context as meaning "upon determining" or "in response to determining" or "upon monitoring [that a described condition or event]" or "in response to monitoring [that a described condition or event]".

[0181] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used merely to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0182] The description of the reference "one embodiment" or "some embodiments" and the like in the present application means that the specific features, structures or characteristics described in connection with the embodiment are included in one or more embodiments of the present application. Therefore, the statements "in one embodiment", "in some embodiments", "in other some embodiments", "in other some embodiments" and the like appearing in different places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized. The terms "include", "contain", "have" and their variants mean "include but not limited to", unless otherwise specifically emphasized.

[0183] The above described embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A multi-functional industrial robot arm path obstacle avoidance optimization method, characterized in that, The multi-functional industrial robot mechanical arm path obstacle avoidance optimization method comprises: Obtain the original control path of the mechanical arm and the standard image, and collect the current depth image of the operation table area through the structured light depth camera; the standard image refers to the image collected when there is no obstacle in the operation table area; According to the pixel value and depth information corresponding to the current depth image and the standard image respectively, the obstacle region in the current depth image is extracted; Based on the preset extension coefficient, the obstacle region is extended to obtain the current image region; Convert the current image region into X-axis coordinates and Y-axis coordinates in the actual environment; Extract the minimum depth value in the first depth information corresponding to the obstacle region; Convert the minimum depth value into Z-axis coordinates; The plane formed by the X-axis coordinates corresponding to the current image region, the Y-axis coordinates corresponding to the current image region, and the Z-axis coordinates is taken as the three-dimensional region top surface; The three-dimensional region top surface is extended to the operation platform surface in the direction perpendicular to the Z-axis direction to obtain a redundant three-dimensional region; the redundant three-dimensional region includes an obstacle three-dimensional region and a redundant region; the redundant region refers to a safety buffer zone around the obstacle region; When the plurality of trajectory points in the original control path are in the redundant three-dimensional region, the projection position of the intersection trajectory point on the surface of the redundant three-dimensional region is extracted; the intersection trajectory point refers to the trajectory point in the redundant three-dimensional region; The intersection trajectory point in the original control path is replaced by the projection position to obtain a target control path, and the mechanical arm is controlled based on the target control path to avoid obstacles.

2. The multi-functional industrial robot arm path obstacle avoidance optimization method of claim 1, wherein, The step of extracting the obstacle region in the current depth image according to the pixel value and depth information corresponding to the current depth image and the standard image respectively comprises: Calculate the pixel difference value between the same pixel positions of the current depth image and the standard image; The pixel position with a pixel difference value greater than a first threshold value is taken as an abnormal pixel position; An abnormal pixel region composed of continuous abnormal pixel positions is extracted; The first depth information of the abnormal pixel region in the current depth image is extracted, and the second depth information of the abnormal pixel region in the standard image is extracted; If the difference between the first depth information and the second depth information exceeds a second threshold value, the abnormal pixel region is taken as the obstacle region.

3. The multi-functional industrial robot arm path obstacle avoidance optimization method of claim 1, wherein, The step of extracting the projection position of the intersection trajectory point on the surface of the redundant three-dimensional region when the plurality of trajectory points in the original control path are in the redundant three-dimensional region comprises: When the plurality of intersection trajectory points are all on the surface of the three-dimensional region, the plurality of intersection trajectory points are taken as the projection position; When the plurality of intersection trajectory points are not all on the surface of the three-dimensional region, the projection position of the intersection trajectory point on the surface of the redundant three-dimensional region is extracted according to the number characteristics of the plurality of intersection trajectory points.

4. The multi-functional industrial robot arm path obstacle avoidance optimization method of claim 3, wherein, The step of extracting the projection position of the intersection trajectory point on the surface of the redundant three-dimensional region when the plurality of intersection trajectory points are not all on the surface of the three-dimensional region according to the number characteristics of the plurality of intersection trajectory points comprises: extracting surface intersection points from the plurality of intersection points; the surface intersection points are intersection points on a surface of the redundant three-dimensional region; if the number of the surface intersection points is two, extracting a first line segment corresponding to the two surface intersection points; extracting a plurality of edge points on a contour of a top surface of the three-dimensional region; constructing a current triangle based on the edge points and the first line segment respectively; the edge points are vertices of the current triangle, and the first line segment is a base of the current triangle; extracting a minimum-area current triangle from the plurality of current triangles; taking two opposite sides of the minimum-area current triangle as projection positions of the surface of the three-dimensional region; the two opposite sides are two sides of the current triangle other than the base; if the number of the surface intersection points exceeds two, sorting the plurality of surface intersection points according to an original order of the intersection points; extracting a second line segment formed by a first surface intersection point and a last surface intersection point; the first surface intersection point is a first intersection point in the plurality of surface intersection points, and the last surface intersection point is a last intersection point in the plurality of surface intersection points; constructing a target triangle based on the edge points and the second line segment respectively; the edge points are vertices of the target triangle, and the second line segment is a base of the target triangle; extracting a minimum-area target triangle from the plurality of target triangles; taking two opposite sides of the minimum-area target triangle as the projection positions of the surface of the three-dimensional region.

5. The multi-functional industrial robot arm path obstacle avoidance optimization method of claim 1, wherein, after the step of constructing the redundant three-dimensional region according to the obstacle region, the method further comprises: if the number of the intersection points is one, controlling the robot arm to avoid the obstacle based on the original control path.

6. The multi-functional industrial robot arm path obstacle avoidance optimization method of claim 1, wherein, after the steps of extracting the first depth information of the abnormal pixel region in the current depth image and extracting the second depth information of the difference pixel region in the standard image, the method further comprises: if a difference between the first depth information and the second depth information does not exceed a second threshold, controlling the robot arm to perform a specified action based on the original control path; in the process of performing the specified action, collecting a real-time depth image of the operation table region by using the structured light depth camera; extracting an obstacle region in the real-time depth image according to pixel values and depth information corresponding to the real-time depth image and the standard image respectively; if no obstacle is identified, continuing to control the robot arm to perform the specified action based on the original control path; if an obstacle is identified, performing an obstacle avoidance process.

7. A multi-functional industrial robot arm path obstacle avoidance optimization device, characterized in that, The multifunctional industrial robot arm path obstacle avoidance optimization device comprises: an acquisition unit configured to acquire an original control path of a robot arm and a standard image, and collect a current depth image of an operation table region by using a structured light depth camera; the standard image is an image collected when no obstacle exists in the operation table region; a first extraction unit configured to extract an obstacle region in the current depth image according to pixel values and depth information corresponding to the current depth image and the standard image respectively; a second extraction unit configured to extract a first intersection point and a last intersection point from the plurality of intersection points; the first intersection point is a first intersection point in the plurality of intersection points, and the last intersection point is a last intersection point in the plurality of intersection points; The construction unit is configured to extend the obstacle region based on a preset extension coefficient to obtain a current image region; convert the current image region into an X-axis coordinate and a Y-axis coordinate in an actual environment; extract a minimum depth value in first depth information corresponding to the obstacle region; convert the minimum depth value into a Z-axis coordinate; take a plane formed by the X-axis coordinate corresponding to the current image region, the Y-axis coordinate corresponding to the current image region, and the Z-axis coordinate as a three-dimensional region top surface; extend the three-dimensional region top surface in a direction perpendicular to the Z-axis to an operation platform surface in a direction of the operation platform to obtain a redundant three-dimensional region; the redundant three-dimensional region includes an obstacle three-dimensional region and a redundant region; the redundant region refers to a safety buffer zone around the obstacle region; The second extraction unit is configured to extract a projection position of a cross trajectory point on a surface of the redundant three-dimensional region when the plurality of trajectory points in the original control path are in the redundant three-dimensional region; the cross trajectory point refers to a trajectory point in the redundant three-dimensional region; The obstacle avoidance unit is configured to replace the cross trajectory point in the original control path with the projection position to obtain a target control path, and control the robot arm to avoid obstacles based on the target control path.

8. A terminal device, comprising: The terminal device comprises a memory, a processor, and a multifunctional industrial robot arm path obstacle avoidance optimization program stored in the memory and executable on the processor, and the multifunctional industrial robot arm path obstacle avoidance optimization program is configured to implement the steps in the multifunctional industrial robot arm path obstacle avoidance optimization method according to any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps in the multifunctional industrial robot arm path obstacle avoidance optimization method according to any one of claims 1 to 6.

Images