A mechanical arm control method based on visual processing

Abstract

The application provides a mechanical arm control method based on visual processing, comprising: acquiring original point cloud data of a target object, processing the original point cloud data by using a three-dimensional reconstruction algorithm to obtain spatial form and volume data of the target object; acquiring a terminal position of a mechanical arm, obtaining a spatial distance by calculating a spatial straight line distance between the terminal position and three-dimensional coordinates of a fragile part; if the spatial distance is less than or equal to a preset distance threshold, it is determined that a current stage is a close distance action stage, and the material attribute and surface roughness of the target object in the close distance action stage are acquired; the material attribute, the surface roughness and the volume data are processed by using a support vector machine to obtain a basic clamping strength; a final control instruction is acquired, and a terminal executor of the mechanical arm is driven to perform a grabbing action on the target object according to the final control instruction.

Description

Technical Field

[0001] This invention relates to the field of information technology, and in particular to a method for controlling a robotic arm based on vision processing. Background Technology

[0002] Automated gripping operations by robotic arms are an indispensable core component of modern intelligent manufacturing and logistics sorting. Current conventional gripping solutions typically treat the entire gripping action as a continuous and homogeneous execution process, relying on globally unified path planning and fixed clamping strategies. This approach ignores the differences in the physical characteristics of the target object at different spatial locations, as well as the drastically different precision requirements of the robotic arm at different approach distances.

[0003] When dealing with complex objects with irregular shapes and fragile parts, such globally unified motion planning can lead to serious execution conflicts. The complexity of the object's shape means that its surface has undulations and vulnerable areas, which means that the robotic arm's trajectory cannot be a simple, unchanging route, but must be broken down into motions and spatial avoidance based on the distance to the object. Coarse spatial trajectory planning can directly lead to unexpected collisions of the end effector during movement, resulting in force imbalance during the gripping phase. For example, when grasping an irregular ceramic artifact with a slender glass handle, if the robotic arm does not adjust its spatial trajectory when approaching the artifact from a distance, it is very likely to directly impact and break the fragile glass handle; and in the final stage of contacting and clamping the artifact, if the system cannot sense the object's material and size and continues to apply conventional mechanical gripping force, it will instantly crush the ceramic body.

[0004] Therefore, how to acquire the three-dimensional shape and material information of an object in real time, scientifically divide the entire grasping process into different action stages, and dynamically generate a movement trajectory that avoids vulnerable parts and matches appropriate clamping force in each stage has become a key issue in achieving non-destructive grasping of complex-shaped targets. Summary of the Invention

[0005] This invention provides a vision-based robotic arm control method, mainly comprising: The process begins by acquiring the raw point cloud data of the target object and processing it using a 3D reconstruction algorithm to obtain its spatial morphology and volume data. Based on this data, the surface curvature features of the target object are extracted. If the surface curvature features exceed a preset curvature threshold, the corresponding area is identified as a vulnerable region, and its 3D coordinates are obtained. The end effector position of the robotic arm is then acquired, and the spatial distance between the end effector position and the 3D coordinates of the vulnerable region is calculated. If this spatial distance exceeds a preset distance threshold, the current operation is considered to be in a long-range motion phase, and avoidance path planning data is generated based on the long-range motion phase and the 3D coordinates of the vulnerable region. If the spatial distance is less than or equal to the preset distance threshold, the current operation is considered to be in a short-range motion phase, and the material properties and surface roughness of the target object in this phase are acquired. A support vector machine is used to process the material properties, surface roughness, and volume data to obtain the basic clamping force. The real-time force state of the robotic arm under this basic clamping force is then acquired. If the real-time force state exceeds a preset force threshold, an overload risk is identified, and a force compensation value is generated based on the difference between the real-time force state and the force threshold. The final gripping force is obtained by adding the base gripping force to the force compensation value. Based on the final gripping force and the obstacle avoidance path planning data, the final control command for the robotic arm is generated. The final control command is then obtained, and the end effector of the robotic arm is driven to perform a grasping action on the target object.

[0006] Furthermore, if the surface curvature feature is greater than a preset curvature threshold, the corresponding area is determined to be a vulnerable part, and the three-dimensional coordinates of the vulnerable part are obtained.

[0007] Furthermore, spatial distance is obtained.

[0008] Furthermore, avoidance path planning data is generated based on the three-dimensional coordinates of the remote action phase and the vulnerable parts.

[0009] Furthermore, the material properties and surface roughness of the target object during the close-range action phase are obtained.

[0010] Furthermore, if there is a risk of clamping overload, a force compensation value is generated based on the difference between the real-time force state and the force threshold.

[0011] Furthermore, the final control commands for the robotic arm are generated based on the final gripping force and obstacle avoidance path planning data.

[0012] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects: This invention discloses a vision-based robotic arm control method, addressing the problem that robotic arms struggle to simultaneously avoid vulnerable areas and precisely control gripping force when grasping complex objects, easily leading to object damage or overload. This invention acquires object shape and volume data through 3D reconstruction and extracts surface curvature features to accurately locate the 3D coordinates of vulnerable areas. Subsequently, it processes data in stages based on the spatial distance between the robotic arm and the vulnerable area. At long distances, it generates avoidance paths for the vulnerable areas; at close distances, it uses a support vector machine combined with material properties, surface roughness, and volume to calculate the basic gripping force. Simultaneously, it monitors the real-time force state and generates force compensation values ​​when there is an overload risk, thus obtaining the final gripping force and control commands. This invention achieves intelligent avoidance of the grasping path and dynamic adaptive adjustment of the gripping force, effectively preventing damage to the target object and significantly improving the safety and reliability of robotic arm grasping operations. Attached Figure Description

[0013] Figure 1 This is a flowchart of a vision-based robotic arm control method according to the present invention. Detailed Implementation

[0014] The technical solutions of the embodiments of the present invention will be clearly and thoroughly described below with reference to the accompanying drawings. The described embodiments are merely some embodiments of the present invention.

[0015] like Figure 1 This embodiment of a vision-based robotic arm control method may specifically include: S101: Acquire the raw point cloud data of the target object, and process the raw point cloud data using a 3D reconstruction algorithm to obtain the spatial shape and volume data of the target object. The raw point cloud data can be acquired and processed quickly or in real time within the workspace of the robotic arm, and the data can accurately reflect the external contour and positional relationships of the target object.

[0016] Raw point cloud data can be acquired and stored before grasping using a depth vision device set up in the working environment. In this application scenario, the raw point cloud data obviously does not include the internal structural information of the target object, but it does include the geometric features of its surface. As we know from the principles of 3D imaging, raw point cloud data is a collection of a large number of spatial points, and in optional embodiments, it is indeed necessary to use this point cloud data to reconstruct a complete 3D model.

[0017] The main purpose of this step is to use the raw point cloud data to mimic the real physical shape of the target object and obtain a high-precision 3D digital model. Surface reconstruction techniques are applied to the acquired point cloud data using 3D reconstruction algorithms, and the resulting spatial morphology and volume data provide a foundation for subsequent grasping strategies. The generation process involves extracting feature points from the point cloud data, performing denoising and registration processing, and constructing a mesh model of the target object. By calculating the spatial region enclosed by the mesh model, the volume data of the target object can be obtained.

[0018] Regarding the 3D reconstruction algorithm described in this application, a preferred embodiment specifically includes the Poisson surface reconstruction algorithm or the Delaunay triangulation algorithm. The preset reconstruction parameters can be obtained through experiments using a large number of real object scans. Appropriate parameters are pre-set so that the spatial shape obtained from the original point cloud data roughly approximates the true physical contour of the target object.

[0019] Considering that different target objects have different surface characteristics and therefore require different scanning parameters, in practical applications, it may not be possible to pre-fix the values ​​of the reconstruction parameters. Instead, it is necessary to calculate the appropriate parameters in real time based on the actual situation of the point cloud data. Therefore, before step S101, the imaging parameters of the scanning device can be obtained first, and the initial reconstruction parameters can be determined based on the imaging parameters.

[0020] S102, Based on the spatial shape and volume data, extract the surface curvature features of the target object. If the surface curvature features are greater than the preset curvature threshold, determine that the corresponding area is a vulnerable part and obtain the three-dimensional coordinates of the vulnerable part.

[0021] Spatial morphology data can be used to determine the degree of geometric changes on the surface of a target object, thereby reducing potential damage to the object during subsequent robotic arm grasping. Although the target object can be classified as a whole using conventional visual recognition methods, experiments have shown that the classification results are very close to the overall attributes of the object, but the accuracy for recognizing local features is not high enough to meet the needs of refined grasping, so further processing is required.

[0022] It should also be noted that the surface curvature features described in this application, as well as the vulnerable parts mentioned later, refer to areas on the surface of the target object that are prone to deformation or damage, such as the edges, tips, or extremely thin protrusions of the object.

[0023] The process of extracting surface curvature features involves calculating the curvature value of each vertex on the surface of a 3D model. The computational model used in this step is pre-defined, extracting feature vectors of the spatial morphology for differential geometric calculations to obtain more accurate local curvature parameters compared to the overall contour.

[0024] Based on the calculated surface curvature characteristics, regions with curvature values ​​greater than a preset curvature threshold are mapped onto the three-dimensional space of the target object. This region is used to characterize predetermined vulnerable locations that the robotic arm needs to avoid during grasping. Vulnerable locations can be a local protrusion of the object, all or part of an edge contour determined based on spatial morphology, or a combination of the above-mentioned vulnerable locations. All of these can characterize the predetermined locations that need to be protected.

[0025] There are at least two ways to achieve this result. The first is to use the curvature calculation formula and a 3D model to obtain a corresponding curvature distribution map. Because the curvature values ​​obtained in this way are more detailed than the overall features, the distribution map usually highlights areas with drastic curvature changes. Then, based on this distribution map, region segmentation techniques can be used to extract the vulnerable parts or their outlines, and then the 3D coordinates of the vulnerable parts or their outlines can be extracted.

[0026] The second implementation method, to improve the accuracy of coordinate extraction, uses raw point cloud data to extract and map vulnerable areas. This involves the following steps: First, high-curvature point sets are segmented from the point cloud data. This can be done automatically using geometric feature-based algorithms or deep learning algorithms. The result is a point cloud sequence of the vulnerable area, or a local 3D model of the vulnerable area. Then, a coordinate transformation matrix is ​​used to map this vulnerable area onto the robot arm's base coordinate system; this is essentially a spatial transformation process. The resulting 3D coordinates are then stored at the corresponding location in the control system.

[0027] S103, obtain the end position of the robotic arm, and obtain the spatial distance by calculating the spatial straight-line distance between the end position and the three-dimensional coordinates of the vulnerable part.

[0028] In this application, the end effector position refers to the real-time coordinates of the robotic arm actuator in three-dimensional space. Therefore, the end effector position is a three-dimensional point with precise positional information. The end effector position can be reconstructed into the kinematic model of the robotic arm or used as a dynamic tracking object. In step S103, the three-dimensional coordinates of the end effector position can be obtained by mapping the angle data of each joint of the robotic arm using a forward kinematics algorithm. This is essentially performing matrix multiplication on the end effector position, and the resulting coordinate data is used for subsequent distance calculations.

[0029] In an optional embodiment, only the distance between the center point of the end effector and the nearest point of the vulnerable area can be calculated. This is because verifying proximity primarily focuses on whether the robotic arm will touch the predetermined vulnerable area, so only the nearest coordinate point needs to be processed, thus avoiding the massive amount of data required for overall calculation. Alternatively, the entire outline of the end effector can be included in the calculation, which verifies whether the robotic arm is completely within a safe space and avoids any unnecessary physical interference.

[0030] After distance calculation is completed, there are several feedback options available, and displaying the distance value through the system interface is neither necessary nor the only option. Displaying the distance result can be an option, decided by the operator. Initially, the system automatically processes the distance data in the background, and only when the operator clicks a monitoring button will the spatial distance be marked in real time on the monitoring screen. The operator can then subjectively judge whether the robotic arm has reached a safe distance.

[0031] S104. If the spatial distance is greater than the preset distance threshold, it is determined that the current stage is a long-distance action stage. Based on the three-dimensional coordinates of the long-distance action stage and the vulnerable part, avoidance path planning data is generated.

[0032] In step S104, a path planning process is introduced to optimize the movement trajectory of the robotic arm. In this embodiment, to distinguish the action strategies at different stages, the state where the spatial distance is greater than a distance threshold is recorded as the long-distance action stage. At this time, the robotic arm is far from the target object, and its main task is to approach the target object quickly and safely, while avoiding the vulnerable parts extracted in step S102.

[0033] The robotic arm's motion trajectory is planned, including adjusting its expected path to ensure sufficient safety margin between the adjusted path and vulnerable areas. This process then determines the corresponding obstacle avoidance path planning data. By adjusting the motion trajectory to better meet safe grasping requirements, the adjusted path parameters can be obtained, thereby improving the safety of subsequent actions.

[0034] It should be noted that in the application scenario of this solution, there may be other obstacles in the workspace, so no matter how the path is adjusted, it will never be an absolutely straight line. However, this step only needs to obtain the path result that maintains a safe distance from the vulnerable parts and is relatively the most efficient.

[0035] There are various planning schemes for obstacle avoidance paths. For example, path planning algorithms based on artificial potential fields or fast expanding random trees can be used. Different algorithms have different accuracy and efficiency. As long as they can meet the actual grasping needs, they are acceptable.

[0036] Furthermore, a crucial task in the path planning process is establishing a repulsive force field for vulnerable areas in three-dimensional space. In some cases, the entire bounding box of the target object can be used to define the avoidance zone, such as treating the entire object as an obstacle. However, in reality, the robotic arm ultimately needs to grasp the object. If the entire object is treated as an obstacle, it may become inaccessible, potentially leading to the inability to generate an effective grasping path.

[0037] To address this issue, a local avoidance strategy can be introduced. This solution utilizes the three-dimensional coordinates of the vulnerable area obtained in the preceding steps to generate a local repulsive force field specifically targeting the vulnerable area in space. This allows the robotic arm to both approach the target object as a whole and precisely avoid the vulnerable area during long-range movements. By using the vulnerable area as the core avoidance target, the efficiency and accuracy of path planning can be improved.

[0038] After determining the avoidance path, the motion parameters of each joint of the robotic arm can be calculated. The generated avoidance path planning data will then serve as the basis for subsequent control. The robotic arm will then perform a series of movements according to this data, eventually smoothly transitioning to a state close to the target object.

[0039] S105, if the spatial distance is less than or equal to a preset distance threshold, it is determined that the current operation is in the close-range action phase, and the material properties and surface roughness of the target object in the close-range action phase are obtained. In this embodiment, when the spatial distance between the end of the robotic arm and the vulnerable part is reduced to within the distance threshold, it indicates that the robotic arm is already very close to the target object. At this time, the action strategy needs to be changed from spatial avoidance to precise physical contact preparation. In order to ensure the stability and safety of subsequent grasping actions, it is necessary to fully understand the surface physical characteristics of the target object.

[0040] There are several ways to obtain material properties and surface roughness. In one possible implementation, a high-precision multispectral vision sensor mounted on the end effector of a robotic arm can be used to scan the surface of a target object at close range. Image recognition algorithms can then be used to analyze the surface's texture features and reflectivity, thereby inferring the material properties and surface roughness. In another alternative embodiment, a miniature tactile sensor mounted on the end effector can be used to obtain the surface friction coefficient during extremely slight, tentative contact, and then calculate the surface roughness. These physical parameters will provide crucial data support for subsequent force calculations.

[0041] S106, a support vector machine (SVM) is used to process material properties, surface roughness, and volume data to obtain the basic clamping force. The main purpose of this step is to use a machine learning model to mimic human experience in grasping different objects, obtaining an initial, relatively safe clamping force value. The output result obtained by using the SVM to comprehensively analyze the multi-dimensional physical parameters is called the basic clamping force.

[0042] The support vector machine model used in this step is pre-trained with a large amount of sample data, and it performs a regression prediction task. Feature vectors from material properties, surface roughness, and the volume data obtained in step S101 are extracted and used for regression calculations to obtain a more accurate clamping force compared to a traditional fixed threshold. The preset model parameters can be adjusted based on a large number of real-world gripping experiments, so that the basic clamping force obtained from these physical parameters is approximately close to the optimal gripping force.

[0043] Given the significant differences in weight distribution and surface friction conditions among different target objects, the clamping strategies employed will inevitably vary. Therefore, in practical applications, it is impossible to pre-determine the clamping force value; instead, a suitable force needs to be calculated based on real-time inference from the support vector machine. By utilizing volume data to roughly estimate the object's mass, and combining this with the frictional characteristics reflected by material properties and surface roughness, the support vector machine can output the smallest possible basic clamping force while ensuring that the object does not slip, thereby reducing compressive damage to the object.

[0044] S107: Obtain the real-time force state of the robotic arm under the basic clamping force. If the real-time force state is greater than a preset force threshold, it is determined that there is a risk of clamping overload. A force compensation value is generated based on the difference between the real-time force state and the force threshold. In this embodiment of the invention, a dynamic optimization process is introduced to adjust the basic clamping force output by the support vector machine model in real time. At the instant the robotic arm contacts the target object according to the basic clamping force, real-time force feedback is obtained through the multi-axis torque sensor built into the end effector.

[0045] Real-time force status is used to characterize the actual physical interaction between the robotic arm and the target object. Although the basic gripping force is a reasonable prediction calculated based on multi-dimensional parameters, unknown local deformations or center of gravity shifts may occur in the actual grasping environment, causing the actual force to exceed expectations. When the real-time force status is detected to be greater than the preset force threshold, the system determines that there is a risk of gripping overload, which may result in crushing or excessively squeezing the target object.

[0046] To address this overload issue, a series of adjustments to the clamping force are needed to ultimately match it with the target object's tolerance. Specifically, a reverse force compensation value is generated by calculating the difference between the real-time force state and the force threshold. This compensation value is essentially a negative feedback adjustment, used to weaken the original base clamping force. Through this adaptive force compensation mechanism, the compliance and safety of the gripping process can be effectively improved, avoiding damage caused by rigid collisions.

[0047] In step S108, the final gripping force is obtained by adding the base gripping force to the force compensation value. Based on this final gripping force and the obstacle avoidance path planning data, the final control command for the robotic arm is generated. After the above compensation process, the initially predicted base gripping force is algebraically added to the dynamically generated force compensation value to obtain the adjusted final gripping force. The final gripping force obtained at this point is more consistent with the current actual physical contact state than the base value obtained in step S106, thus meeting the needs of precise and non-destructive gripping.

[0048] It should also be noted that the final control command described in this application is a comprehensive control signal that integrates spatial motion trajectory and end-effector force. The system uses the avoidance path planning data generated in step S104 as the control input for the position loop, ensuring that the robotic arm perfectly avoids vulnerable parts in space; at the same time, it uses the final gripping force as the control input for the torque loop, ensuring that the end effector applies just the right amount of force when contacting the object. By encapsulating these two parts of data through a low-level protocol, a final control command that can directly drive the robotic arm's servo motor is generated.

[0049] S109: Obtain the final control command and drive the end effector of the robotic arm to perform a grasping action on the target object according to the final control command. After completing all path planning and force calculations, the control system sends the final control command to the lower-level controller of the robotic arm. The lower-level controller parses the command and converts it into current and voltage signals for each joint motor, thereby driving the robotic arm to move smoothly along a predetermined safe path.

[0050] When the end effector reaches the designated gripping position of the target object, it closes the gripper or activates the suction cup according to the final gripping force, completing the gripping action on the target object. In optional embodiments, the device executing this method can also monitor the entire gripping process in real time and allow the operator to intervene manually when necessary. For example, the operator can interrupt the gripping through the control panel or fine-tune the final control command through the teach pendant. Through this combination of subjective and objective control, efficient and safe automated gripping of the target object is achieved.

[0051] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and additions without departing from the principle of the present invention, and these improvements and additions should also be considered within the scope of protection of the present invention.

Claims

1. A robotic arm control method based on vision processing, characterized in that, include: The original point cloud data of the target object is acquired, and the original point cloud data is processed by a 3D reconstruction algorithm to obtain the spatial morphology data and volume data of the target object. Based on the spatial morphology data and the volume data, the surface curvature features of the target object are extracted. If the surface curvature features are greater than a preset curvature threshold, the corresponding area is determined to be a vulnerable part, and the three-dimensional coordinates of the vulnerable part are obtained. The end effector position of the robotic arm is obtained, and the spatial distance is obtained by calculating the straight-line distance between the end effector position and the three-dimensional coordinates of the vulnerable part. If the spatial distance is greater than a preset distance threshold, it is determined that the current stage is a long-distance action, and avoidance path planning data is generated based on the three-dimensional coordinates of the vulnerable part and the long-distance action stage. If the spatial distance is less than or equal to a preset distance threshold, it is determined that the current action phase is in the close-range action phase, and the material properties and surface roughness of the target object under the close-range action phase are obtained. The material properties, surface roughness, and volume data are processed using a support vector machine to obtain the basic clamping force. The real-time force state of the robotic arm under the basic clamping force is obtained. If the real-time force state is greater than the preset force threshold, it is determined that there is a risk of clamping overload. A force compensation value is generated based on the difference between the real-time force state and the force threshold. The final clamping force is obtained by calculating the sum of the basic clamping force and the force compensation value. The final control command of the robotic arm is generated based on the final clamping force and the avoidance path planning data. Obtain the final control command, and drive the end effector of the robotic arm to perform a grasping action on the target object according to the final control command.

2. The vision-based robotic arm control method according to claim 1, characterized in that, The step involves extracting the surface curvature features of the target object based on the spatial morphology data and the volume data. If the surface curvature features are greater than a preset curvature threshold, the corresponding region is determined to be a vulnerable area, and the three-dimensional coordinates of the vulnerable area are obtained. This includes: Extract the three-dimensional contour node data from the spatial morphology data; Construct a surface mesh model of the target object based on the three-dimensional contour node data; Calculate the normal vector data of each mesh node in the surface mesh model; The surface curvature characteristics of the target object are calculated based on the normal vector data and the volume data. Compare the surface curvature features with a preset curvature threshold; If the surface curvature feature is greater than a preset curvature threshold, extract the mesh node corresponding to the surface curvature feature; The region where the mesh node corresponding to the surface curvature feature is located is identified as a vulnerable area. The three-dimensional coordinates of the vulnerable parts are extracted from the surface mesh model to obtain the three-dimensional coordinates of the vulnerable parts.

3. The vision-based robotic arm control method according to claim 1, characterized in that, The process of obtaining the end effector position of the robotic arm, by calculating the spatial distance between the end effector position and the three-dimensional coordinates of the vulnerable part, includes: Obtain the kinematic model data of the robotic arm; The current pose matrix of the end effector of the robotic arm is extracted based on the kinematic model data; The three-dimensional coordinates of the end effector position of the robotic arm are resolved from the current pose matrix; Obtain the center coordinates of the vulnerable part in the three-dimensional coordinate system; Construct a three-dimensional spatial coordinate system between the three-dimensional coordinate point of the end position and the center coordinate point of the vulnerable part; In the three-dimensional spatial coordinate system, calculate the Euclidean distance between the three-dimensional coordinate point of the end position and the center coordinate point of the vulnerable part; The Euclidean distance is determined as the spatial straight-line distance between the end position and the three-dimensional coordinates of the vulnerable part, and the spatial distance is obtained.

4. The vision-based robotic arm control method according to claim 1, characterized in that, If the spatial distance is greater than a preset distance threshold, it is determined that the current operation is in a long-distance action phase. Based on the long-distance action phase and the three-dimensional coordinates of the vulnerable part, avoidance path planning data is generated, including: Extract the spatial distance and compare it with a preset distance threshold; If the spatial distance is greater than a preset distance threshold, a remote action phase identifier is generated, and it is determined that the current stage is a remote action phase. Obtain the initial motion trajectory data of the robotic arm under the remote action stage identifier; Extract the three-dimensional coordinates of the vulnerable part, and construct a three-dimensional bounding box model of the vulnerable part based on the three-dimensional coordinates of the vulnerable part; The three-dimensional bounding box model is expanded outward by a preset safety margin to obtain a safe avoidance area model. Spatial interference detection is performed between the initial motion trajectory data and the safe avoidance area model; If the initial motion trajectory data and the safe avoidance area model have spatial interference, extract the interference node data; Based on the interference node data and the boundary coordinates of the safe avoidance area model, the avoidance guidance vector is calculated using the artificial potential field algorithm; Based on the avoidance guidance vector, the coordinate offset processing is performed on the interference node data in the initial motion trajectory data to obtain the updated motion trajectory node data; The avoidance path planning data is generated based on the updated motion trajectory node data.

5. The vision-based robotic arm control method according to claim 1, characterized in that, If the spatial distance is less than or equal to a preset distance threshold, it is determined that the current action is in a close-range phase. The material properties and surface roughness of the target object during the close-range action phase are then obtained, including: If the spatial distance is less than or equal to a preset distance threshold, a close-range action phase identifier is generated, and it is determined that the current position is in the close-range action phase. The hyperspectral camera and laser profilometer mounted on the end effector of the robotic arm are activated according to the near-field action phase indicator. The surface spectral reflectance data of the target object are acquired using the hyperspectral camera. The material properties of the target object are extracted by matching the surface spectral reflectance data with a preset material spectral database. The laser profilometer scans the surface of the target object to obtain surface micro-undulation profile data. Calculate the arithmetic mean deviation of the surface micro-undulation profile data; The arithmetic mean deviation value is determined to be the surface roughness of the target object.

6. The vision-based robotic arm control method according to claim 1, characterized in that, The process of using a support vector machine to process the material properties, surface roughness, and volume data to obtain the basic clamping force includes: Extract the material properties, surface roughness, and volume data; The material properties, surface roughness, and volume data are normalized to obtain standard material feature vectors, standard roughness feature vectors, and standard volume feature vectors. The standard material feature vector, the standard roughness feature vector, and the standard volume feature vector are concatenated to construct a comprehensive feature matrix; Input the comprehensive feature matrix into the pre-trained support vector machine regression model; High-dimensional feature data is obtained by performing nonlinear mapping on the comprehensive feature matrix through the kernel function in the support vector machine regression model. The high-dimensional feature data is regressed using the hyperplane parameters in the support vector machine regression model to output a predicted intensity value. The predicted force value is determined as the base clamping force.

7. The vision-based robotic arm control method according to claim 1, characterized in that, The process involves acquiring the real-time force state of the robotic arm under the base clamping force. If the real-time force state exceeds a preset force threshold, an overload risk is identified. A force compensation value is generated based on the difference between the real-time force state and the force threshold, including: Send the basic clamping force command to the end effector of the robotic arm; The three-dimensional force data and three-dimensional torque data of the robotic arm under the basic clamping force are collected by the six-axis torque sensor on the end effector. The three-dimensional force data and the three-dimensional torque data are filtered to obtain the real-time force state. Extract the resultant force value from the real-time stress state and compare the resultant force value with a preset stress threshold. If the resultant force value is greater than the preset force threshold, an overload risk flag is generated, indicating that there is an overload risk. Calculate the difference between the resultant force value and the force threshold to obtain the force overshoot data; Obtain the elastic modulus coefficient corresponding to the material properties of the target object; Input the force overshoot data and the elastic modulus coefficient into the preset proportional-integral-derivative control algorithm; The reverse adjustment force value is calculated using the proportional-integral-derivative control algorithm. The value of the reverse adjustment force is determined as the force compensation value.

8. The vision-based robotic arm control method according to claim 1, characterized in that, The final clamping force is obtained by calculating the sum of the base clamping force and the force compensation value. Based on the final clamping force and the avoidance path planning data, the final control command for the robotic arm is generated, including: Extract the basic clamping force and the force compensation value; The final clamping force is obtained by algebraically summing the basic clamping force and the force compensation value. Extract the spatial coordinates and attitude angle data of each path node from the obstacle avoidance path planning data; The final clamping force is added to the target grasping node in the avoidance path planning data to generate a path node sequence with force constraints; The inverse kinematics solution is performed on the path node sequence with force constraints to obtain the rotation sequence data of each joint of the robotic arm; The final control commands for the robotic arm are generated based on the rotation sequence data.

9. The vision-based robotic arm control method according to claim 1, characterized in that, The step of obtaining the final control command and driving the end effector of the robotic arm to perform a grasping action on the target object according to the final control command includes: Extract the rotation sequence data of each joint and the final clamping force of the target grasping node from the final control command; Send the rotation sequence data of each joint to the underlying servo driver of the robotic arm; The underlying servo driver controls each joint of the robotic arm to move according to the rotation sequence data; When the robotic arm moves to the target grasping node, the final gripping force is sent to the gripper controller of the end effector; The gripper controller drives the end effector to close according to the final gripping force; Complete the grasping action on the target object.

Images