Mechanical arm grabbing method and system based on image processing

Abstract

The invention relates to the technical field of intelligent mechanical arm control, in particular to a mechanical arm grabbing method and system based on image processing, and the method comprises the steps: collecting an image of a target object needing to be grabbed by a mechanical arm through a visual sensor, carrying out the preprocessing and multi-scale decomposition of the collected image of the target object, screening a region of a potential grabbing point, and carrying out the recognition of the target object; and generating a capture point candidate area map. According to the invention, through processing and multi-scale decomposition of the target object image, noise is eliminated, an illumination area is balanced, a clearer target feature image is generated, and environmental adaptability is enhanced. And in combination with texture change detection and gradient analysis, the feature intensity and boundary characteristics of the object are accurately extracted, and more accurate grabbing point screening is realized. Furthermore, the three-dimensional surface form projection is in a two-dimensional form, and the stability and operability of the grabbing area are improved by optimizing the boundary symmetry and curvature fluctuation.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robotic arm control, and in particular to a robotic arm grasping method and system based on image processing. Background Art

[0002] Intelligent robotic arm control technology is a multidisciplinary field that integrates robotics, artificial intelligence, automatic control, sensor technology, and computer science. It aims to automate and intelligently control complex tasks through precise and efficient robotic arm control. Core technologies include motion planning, path optimization, force control, vision guidance, and human-machine interaction.

[0003] Among them, the robotic arm grasping method focuses on how the robotic arm can identify target objects in different environments, calculate the optimal grasping point, and perform grasping operations safely and efficiently. Its uses include material handling in automated production lines, cargo picking in warehousing and logistics, and flexible and complex grasping tasks in the medical and service industries.

[0004] Existing technologies do not adequately address uneven lighting and noise interference during image acquisition of target objects, which can easily lead to a decrease in image quality and affect the accuracy of grasping point extraction. The feature extraction method is relatively simple, and its ability to recognize objects with complex textures and irregular shapes is limited, making it difficult to ensure the accurate positioning of grasping points. Insufficient three-dimensional morphological analysis and optimization cannot effectively handle boundary asymmetry and curvature changes, resulting in poor stability in the grasping area, which may affect the success rate of grasping. In terms of dynamic characteristics, the motion information of the target object is not fully utilized for stability sorting, which can easily cause offsets or errors in the grasping action. The grasping force distribution lacks in-depth analysis of surface characteristics, and there are deficiencies in the reliability of grasping under complex surface conditions. In addition, path planning fails to combine the dynamic changes of the grasping point in real time, which can easily lead to a disconnect between the grasping action and the movement of the robotic arm, reducing the efficiency and accuracy of the operation. These problems limit the application scope and performance of robotic arm grasping technology in complex and changing environments. Summary of the Invention

[0005] In order to solve the technical problems existing in the prior art, the embodiments of the present invention provide a robotic arm grasping method and device based on image processing. The technical solution is as follows:

[0006] On the one hand, a robotic arm grasping method based on image processing is provided, comprising the following steps:

[0007] S1: The robot collects images of the target object to be grasped by the robot arm through the visual sensor, preprocesses and decomposes the collected target object images at multiple scales, screens the areas of potential grasping points, and generates a grasping point candidate area map;

[0008] S2: extracting geometric features of the grasping point candidate region map, projecting the three-dimensional geometric shape of the candidate region into a two-dimensional image, analyzing and optimizing the boundary symmetry of the candidate region in the two-dimensional image, and generating a geometrically optimized grasping candidate region map;

[0009] S3: Performing dynamic characteristic analysis on the geometric optimization grasping candidate area map to determine the motion stability of the target object, sorting the grasping points according to the motion stability of the target object, and generating a dynamic sorting grasping point map;

[0010] S4: performing frequency domain analysis on surface texture features of the region to which the grasping points in the dynamically sorted grasping point graph belong, adjusting the contact direction and force distribution of the grasping points, and generating surface optimized grasping point distribution information;

[0011] S5: Based on the surface optimization grasping point distribution information, combined with the robot arm path planning, dynamic path matching is performed on the grasping points, and the corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time to generate a dynamic grasping path for the robot arm.

[0012] The present invention has improvements in that the grasping point candidate area map includes potential grasping points distributed in the target object surface texture uniform area, edge stable area, and color contrast area, the geometric optimization grasping candidate area map includes the area with boundary symmetry deviation within a preset range, and the area with curvature change amplitude within a preset threshold range, the dynamic sorting grasping point map includes grasping points in the static area, grasping points whose movement speed matches the preset threshold, and grasping points whose movement direction changes match the preset range, the surface optimization grasping point distribution information includes grasping points whose normal direction is parallel to the target grasping path, grasping points whose surface roughness value matches the preset threshold, and grasping points whose contact direction is adjusted by force distribution, and the dynamic grasping path of the robotic arm includes real-time matching of the grasping point position and the robotic arm movement trajectory, the path of aligning the grasping angle with the coordinate system, and the trajectory adjustment of the grasping point position.

[0013] The improvement of the present invention is that the image of the target object to be grasped by the robotic arm is collected by a visual sensor, and the collected target object image is preprocessed and multi-scale decomposed to screen the area of ​​potential grasping points. The specific steps of generating a grasping point candidate area map are as follows:

[0014] S101: Based on the target object image captured by the visual sensor, noise is removed by analyzing the pixel intensity distribution, and brightness is adjusted to balance the illumination area to obtain a processed target object image;

[0015] S102: Decomposing the processed target object image into multiple layers and detecting texture changes layer by layer, extracting color difference areas using pixel contrast values, accumulating gradient changes by obtaining edge strength, and generating a feature intensity map of the target object;

[0016] S103: Based on the feature intensity map of the target object, the curvature change trend is detected to verify the integrity of the boundary area, the directional features of the directional gradient distribution marked area are extracted, the potential grasping point area is screened, and a grasping point candidate area map is generated.

[0017] The present invention has the following improvements: extracting geometric features of the grab point candidate area map, projecting the three-dimensional geometric shape of the candidate area into a two-dimensional image, analyzing and optimizing the boundary symmetry of the candidate area in the two-dimensional image, and generating a geometrically optimized grab candidate area map.

[0018] S201: extracting the three-dimensional surface morphology of the candidate area in the grasp point candidate area map, decomposing the area into evenly distributed dot arrays by gridding, reconstructing a two-dimensional representation of the three-dimensional morphology in a projection plane using height values ​​of the dot arrays, and generating a two-dimensional candidate area image;

[0019] S202: Based on the two-dimensional candidate region image, the position of the symmetry axis of each candidate region is detected, the number distribution deviation of the boundary points on both sides of the symmetry axis is counted, the fluctuation amplitude of the boundary curvature is calculated, and the region whose curvature fluctuation amplitude is within the fluctuation threshold is selected to obtain a symmetry optimization region map;

[0020] S203: Based on the symmetry optimization region map, analyze the distribution density of the grasping points in the remaining region, optimize the spatial position relationship of the grasping points in the region by adjusting the spatial arrangement, and generate a geometric optimization grasping candidate region map.

[0021] The present invention has the following improvements: for the statistical distribution deviation of the number of boundary points on both sides of the symmetry axis, the formula is adopted:

[0022]

[0023] Calculate the fluctuation amplitude of the boundary curvature ;

[0024] in, It is The curvature value of the boundary point, is the mean curvature value, is the total number of boundary points.

[0025] The present invention is improved in that the specific steps of performing dynamic characteristic analysis on the geometric optimization grasping candidate area map, judging the motion stability of the target object, sorting the grasping points according to the motion stability of the target object, and generating a dynamically sorted grasping point map are as follows:

[0026] S301: Based on the geometric optimization grasping candidate area map, using a continuous image sequence of objects in the scene to extract optical flow information, obtain the motion speed of the area where the grasping point is located, extract the dynamic properties of the area by comparing the speed values, and obtain a motion speed information map;

[0027] S302: Based on the motion speed information graph, according to the motion direction distribution of the grasping point area, analyzing the directional consistency of the velocity vector in each area, extracting the stability characteristics of the target motion by statistically analyzing the direction differences, and obtaining a motion stability analysis graph;

[0028] S303: Based on the motion stability analysis diagram, the grasping points are sorted according to the stability level, and the sorting priority is adjusted in combination with the motion speed and direction to generate a dynamic sorting grasping point diagram.

[0029] The present invention is improved in that the specific steps of performing frequency domain analysis on the surface texture characteristics of the area to which the grasping points of the dynamically sorted grasping point map belong, adjusting the contact direction and force distribution of the grasping points, and generating surface optimized grasping point distribution information are as follows:

[0030] S401: extracting surface texture information of the area to which the grab points belong based on the dynamically sorted grab point map, performing frequency domain characteristic analysis, separating the spatial distribution characteristics and directional information of the surface texture, and obtaining a surface texture feature map;

[0031] S402: Based on the surface texture feature map, the surface normal direction of the grasped area is obtained, the normal direction distribution is obtained by analyzing the phase consistency of the surface texture, the surface roughness value is measured and compared with a preset roughness threshold, and the area that meets the roughness requirement is screened to obtain a surface optimization feature map;

[0032] S403: Based on the surface optimization feature map, the contact direction and force distribution of the grasping point are optimized, and the force range of the grasping point is evenly distributed by adjusting the angle between the normal direction and the contact surface of the robotic arm gripper to generate surface optimized grasping point distribution information.

[0033] The present invention is improved in that, based on the surface optimization grasping point distribution information, dynamic path matching of the grasping points is performed in combination with the robot arm path planning, and the corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time. The specific steps of generating the dynamic grasping path of the robot arm are as follows:

[0034] S501: Analyzing the spatial positions of the grasping points based on the surface optimization grasping point distribution information, mapping the coordinates of the grasping points to the manipulator workspace, dynamically matching the grasping points in combination with the manipulator motion trajectory, and generating grasping point path matching data;

[0035] S502: Analyze the spatial relationship between the grasping point and the center of gravity of the object based on the grasping point path matching data, predict the morphological changes that will occur to the object during the grasping action, adjust the robot arm path planning, and obtain a morphological change adjustment path diagram;

[0036] S503: Based on the morphological change, the path diagram is adjusted to optimize the grasping direction and angle of the robot arm, and the adjusted grasping trajectory is output to generate a dynamic grasping path of the robot arm.

[0037] The present invention has the following improvements: to predict the shape change of an object during the grasping action, the formula is used:

[0038]

[0039] Calculate the displacement vector of the center of mass ;

[0040] in, is the total mass of the object, Indicates the effect on The magnitude and direction of the force at each gripping point, is the spatial position vector of the grasping point, is the adjustment coefficient vector, is the total number of grab points.

[0041] A robotic arm grasping system based on image processing, the system comprising:

[0042] The image processing module collects images of target objects using the visual sensor, adjusts brightness and contrast to balance the light distribution, screens the target area by analyzing layer texture changes, edge strength, and color contrast, marks potential grasping points, and generates a grasping point candidate area map;

[0043] The geometric characteristic analysis module extracts the three-dimensional geometric features of the candidate areas based on the grasping point candidate area map and projects them into a two-dimensional image. The module determines the symmetry of the area by analyzing the distribution deviation of the boundary points on both sides of the symmetry axis, detects the fluctuation amplitude of the boundary curvature, and generates a geometric optimization grasping candidate area map by selecting areas where both the symmetry deviation and the fluctuation amplitude meet the preset fluctuation threshold.

[0044] The dynamic sorting module extracts a continuous image sequence of objects in the scene based on the geometric optimization grasping candidate area map, calculates the movement speed and direction of the grasping point area through optical flow analysis, determines the motion stability of the target object, sorts the priority of the grasping points, and generates a dynamic sorting grasping point map;

[0045] The path optimization module extracts the surface texture features of the area to which the grasping points belong based on the dynamically sorted grasping point map, measures the normal direction and roughness in the texture features, optimizes and adjusts the grasping points that exceed the roughness threshold, and dynamically matches the grasping points in combination with the robot arm motion trajectory. The corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time to generate a dynamic grasping path for the robot arm.

[0046] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least: by processing the target object image and multi-scale decomposition, noise is eliminated, the lighting area is balanced, a clearer target feature image is generated, and environmental adaptability is enhanced. Combined with texture change detection and gradient analysis, the characteristic intensity and boundary characteristics of the object are accurately extracted, and more accurate grasping point screening is achieved. Further utilizing the three-dimensional surface morphology projection into a two-dimensional form, the stability and operability of the grasping area are improved by optimizing the boundary symmetry and curvature fluctuation. Dynamic characteristic analysis captures the motion characteristics within the area through optical flow information, determines the stability of the grasping area and optimizes the sorting, making the grasping action more efficient and safe. Surface texture frequency domain analysis is combined with contact force distribution optimization to adapt to complex surface conditions and improve the reliability of grasping contact. Combined with path planning, the correspondence between the grasping point and the robot arm movement is dynamically adjusted to further optimize the real-time and accuracy of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0048] Figure 1 A flowchart of a robotic arm grasping method based on image processing is proposed for the present invention;

[0049] Figure 2 This is a detailed flow chart of step S1 of the present invention;

[0050] Figure 3 This is a schematic diagram of a detailed process of step S2 of the present invention;

[0051] Figure 4 This is a detailed flow chart of step S3 of the present invention;

[0052] Figure 5 This is a detailed flow chart of step S4 of the present invention;

[0053] Figure 6 This is a schematic diagram of a detailed process of step S5 of the present invention;

[0054] Figure 7The present invention proposes a module diagram of a robotic arm grasping system based on image processing. DETAILED DESCRIPTION

[0055] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0056] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations, or explanations. Any embodiment or design described as an "exemplary" in the present invention should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of the word "exemplary" is intended to present concepts in a concrete manner. Furthermore, in the embodiments of the present invention, "and / or" can mean both or either of the two.

[0057] In the embodiments of the present invention, the terms "image" and "picture" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same. The terms "of," "corresponding," and "corresponding" may sometimes be used interchangeably. It should be noted that, when the distinction is not emphasized, the meanings they convey are the same.

[0058] In the embodiments of the present invention, sometimes a subscript such as W1 may be written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are the same.

[0059] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0060] The embodiment of the present invention provides a robotic arm grasping method based on image processing, comprising the following steps:

[0061] S1: The robot collects images of the target object to be grasped by the robot arm through the visual sensor, preprocesses and decomposes the collected target object images at multiple scales, screens the areas of potential grasping points, and generates a grasping point candidate area map;

[0062] S2: Extract the geometric features of the grasp point candidate region map, project the 3D geometric shape of the candidate region into a 2D image, analyze and optimize the boundary symmetry of the candidate region in the 2D image, and generate a geometrically optimized grasp candidate region map;

[0063] S3: Analyze the dynamic characteristics of the geometric optimization grasping candidate area map to determine the motion stability of the target object, sort the grasping points according to the motion stability of the target object, and generate a dynamic sorting grasping point map;

[0064] S4: Perform frequency domain analysis on the surface texture features of the area where the grasping points in the dynamic sorting grasping point map belong, adjust the contact direction and force distribution of the grasping points, and generate surface optimized grasping point distribution information;

[0065] S5: Based on the surface optimization grasping point distribution information, combined with the robot arm path planning, the grasping points are dynamically matched, and the corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time to generate the dynamic grasping path of the robot arm.

[0066] The grasping point candidate area map includes potential grasping points distributed in the target object's surface texture uniform area, edge stable area, and color contrast area. The geometric optimization grasping candidate area map includes the area with boundary symmetry deviation within the preset range and the area with curvature change amplitude within the preset threshold range. The dynamic sorting grasping point map includes grasping points in the static area, grasping points whose movement speed matches the preset threshold, and grasping points whose movement direction changes match the preset range. The surface optimization grasping point distribution information includes grasping points whose normal direction is parallel to the target grasping path, grasping points whose surface roughness value matches the preset threshold, and grasping points whose contact direction is adjusted by force distribution. The dynamic grasping path of the robot arm includes real-time matching of the grasping point position and the robot arm's motion trajectory, the path of aligning the grasping angle with the coordinate system, and the trajectory adjustment of the grasping point position.

[0067] See also Figure 2 , the specific steps for collecting images of the target object that the robot arm needs to grasp through the visual sensor, preprocessing and multi-scale decomposition of the collected target object images, screening the areas of potential grasping points, and generating the grasping point candidate area map are as follows:

[0068] S101: Based on the target object image captured by the visual sensor, noise is removed by analyzing the pixel intensity distribution, and brightness is adjusted to balance the illumination area to obtain a processed target object image;

[0069] Based on the target object image collected by the visual sensor, the pixel intensity distribution in the image is obtained. First, a denoising operation is performed on the image. The median filtering method is used to scan the grayscale values ​​of each pixel point and its surrounding neighborhood pixels one by one, and the median of the pixels in the neighborhood is selected to replace the target pixel grayscale value to eliminate the interference of sudden noise. After completing the denoising process, a noise-free image is obtained. The denoised image is brightness adjusted using MATLAB software, and the brightness range of the image is stretched by histogram equalization to ensure that the dark areas in the image become brighter and the bright areas remain unchanged. After completing the brightness adjustment, the local contrast of each area is calculated based on a fixed window. The image is clear by adjusting the local grayscale value difference. Finally, the processed image data is combined to obtain the processed target object image.

[0070] S102: Decomposing the processed target object image into multiple layers and detecting texture changes layer by layer, extracting color difference areas using pixel contrast values, accumulating gradient changes by obtaining edge strength, and generating a feature intensity map of the target object;

[0071] Based on the processed target object image, the OpenCV library is used in the Python environment to perform multi-scale decomposition of the image using the Laplacian pyramid decomposition method, generating multiple layers of different resolutions. The grayscale difference between each pixel in the layer is obtained layer by layer to extract the texture features of the image, and the detailed changes in the texture in each layer are analyzed. The extracted texture features are superimposed to form a texture change feature map. The color information is obtained by pixel-by-pixel scanning, the color differences between adjacent pixels are compared, and the areas with obvious color changes are marked. The information of these marks is combined to generate a color difference map. Then, the Sobel edge detection method is used to extract the edge strength using the gradient changes in the horizontal and vertical directions. The edge strength features of each layer are accumulated to form an edge feature map. Finally, the texture change feature map, color difference map and edge feature map are superimposed to generate a feature intensity map of the target object.

[0072] S103: Based on the feature intensity map of the target object, the curvature change trend is detected to verify the integrity of the boundary area, the directional features of the directional gradient distribution marked area are extracted, the potential grasping point area is screened, and a grasping point candidate area map is generated;

[0073] Based on the feature intensity map of the target object, the curvature analysis of the boundary area of ​​the image is performed, and the boundary analysis in MATLAB is used to process the boundary area. The boundary pixel points are calculated point by point using the second-order derivative method to extract the curvature value. The angle change of the adjacent pixels of each boundary point is calculated to obtain the amplitude of the curvature change on the boundary. The boundary areas with small curvature change and high continuity are marked to ensure the boundary integrity of these areas. The main directional characteristics of the marked boundary areas are analyzed by the directional gradient histogram in MATLAB, and the frequency of directional gradient changes on the boundary is counted. The areas with high directional consistency are extracted as potential grasping point areas. Combined with the spatial positions of these areas, the specific positions of the potential grasping points are marked to generate a grasping point candidate area map.

[0074] See also Figure 3 , extract the geometric features of the grasping point candidate area map, project the three-dimensional geometric shape of the candidate area into a two-dimensional image, analyze and optimize the boundary symmetry of the candidate area in the two-dimensional image, and generate the geometrically optimized grasping candidate area map in the following specific steps:

[0075] S201: extracting the three-dimensional surface morphology of the candidate region in the grasp point candidate region map, decomposing the region into a uniformly distributed dot matrix by gridding, reconstructing a two-dimensional representation of the three-dimensional morphology in a projection plane using the height values ​​of the dot matrix, and generating a two-dimensional candidate region image;

[0076] The three-dimensional surface morphology of the candidate regions in the grasp point candidate region map is extracted. Each candidate region is divided into an equidistant array using a gridding method. The coordinate values ​​of the points in three-dimensional space are generated by spatially mapping each pixel in the two-dimensional image. The height value of each point is obtained directly from the image grayscale value mapping method. The height data is interpolated to compensate for the influence of discrete distribution, thus generating a complete three-dimensional point cloud data. A filtering method is applied to the height values ​​of the three-dimensional point cloud data. Discrete and missing points are filled using the fillmissing function in MATLAB. Based on the three-dimensional point cloud model, principal component analysis (PCA) is used to reconstruct the spatial characteristics of the point cloud. The direction with the most concentrated distribution is projected onto a two-dimensional plane, and a two-dimensional representation is generated for the projected plane data. The boundary and detail noise in the projection result are processed using data smoothing methods to ensure that the final projection result accurately reflects the three-dimensional morphological characteristics of the candidate region, and finally a two-dimensional candidate region image is generated.

[0077] S202: Based on the two-dimensional candidate region image, the position of the symmetry axis of each candidate region is detected, the number distribution deviation of the boundary points on both sides of the symmetry axis is counted, the fluctuation amplitude of the boundary curvature is calculated, and the regions whose curvature fluctuation amplitude is within the fluctuation threshold are selected to obtain a symmetry optimization region map;

[0078] To detect the position of the symmetry axis of each candidate area, we first perform boundary extraction on the two-dimensional candidate area image to generate a set of closed boundary curves. By performing point-by-point detection on the boundary curves, we calculate the coordinate set of the boundary center points, determine the distribution range of all center points, and use the least squares method to fit a straight line passing through these center points as the symmetry axis.

[0079] For the statistical distribution deviation of the number of boundary points on both sides of the symmetry axis, the formula is used:

[0080]

[0081] Calculate the fluctuation amplitude of the boundary curvature ;

[0082] in, It is The curvature value of each boundary point is obtained by detecting the rate of change of the normal direction of the boundary point, and is calculated using the change angle of the normal direction and the distance between adjacent points. The formula is: , Indicates the change in the normal direction angle, which is measured by the geometric shape of the boundary point. Represents the distance between adjacent boundary points, calculated by the Euclidean distance between image coordinates, is the mean curvature value, which represents the average level of the overall curvature of the boundary. It is obtained by calculating the sum of the curvature values ​​of all points and dividing it by the number of points. The formula is: , is the total number of boundary points, which is obtained by performing contour extraction on the pixels in the boundary area and then counting them. The bwboundaries function of MATLAB is used to generate the boundary pixel point set.

[0083] For example, if the normal angle from boundary point 1 to point 2 changes to , the distance is 0.5mm, then the curvature value of point 1 is , if the boundary contains 5 points, the curvature values ​​are ,but:

[0084] Mean curvature ;

[0085] Fluctuation range ;

[0086] The results show that the fluctuation range , and compare it with the preset fluctuation threshold. If the fluctuation amplitude of the region is less than or equal to the fluctuation threshold, the region is retained; otherwise, it is eliminated. After screening the curvature fluctuation amplitude of all candidate regions, the qualified regions are counted and their spatial positions are re-marked. The symmetry optimization region map is generated using MATLAB.

[0087] S203: Based on the symmetry optimization region map, analyze the distribution density of the grasping points in the remaining region, optimize the spatial position relationship of the grasping points in the region by adjusting the spatial arrangement, and generate a geometrically optimized grasping candidate region map;

[0088] Based on the symmetry-optimized region map, the density of grasping points in each remaining region is analyzed. The spatial coordinates of all grasping points within the region are first extracted, and the two-dimensional or three-dimensional coordinate information of each grasping point is recorded point by point. Euclidean distances are then calculated between these coordinate points to generate a distance matrix between grasping points. By analyzing the numerical distribution of the distance matrix, regions with excessively large inter-point distances are detected and marked as low-density regions. Within these marked low-density regions, point clustering is redistributed using MATLAB. By re-dividing the cluster centers, grasping points are moved closer to more concentrated locations to ensure spatial uniformity. During the point redistribution process, a predefined minimum safe distance is set to ensure a safe distance between grasping points and the region boundary. Boundary distance constraints are then used to redistribute grasping points close to the boundary. These points are then adjusted based on the constraints. In the case of adjacent points being too close, grasping point positions are re-adjusted to ensure that all points are within a preset distance range, ultimately resulting in a reasonably uniform point distribution. Finally, all optimized grasping points are rearranged, the positional relationship of each grasping point in space is recorded and marked, the grasping point distribution information is output, and a geometrically optimized grasping candidate area map is generated.

[0089] See also Figure 4 , analyze the dynamic characteristics of the geometric optimization grasping candidate area map, judge the motion stability of the target object, sort the grasping points according to the motion stability of the target object, and generate the dynamic sorting grasping point map. The specific steps are as follows:

[0090] S301: Based on geometric optimization, a candidate region map is obtained, and optical flow information is extracted using a continuous image sequence of objects in the scene. The motion speed of the region where the grasp point is located is obtained, and the dynamic properties of the region are extracted by comparing the speed values ​​to obtain a motion speed information map.

[0091] Based on the geometric optimization of the grasping candidate area map, the continuous image sequence of the objects in the scene is used to extract the optical flow information. First, the target area is continuously captured by frame, and the motion information of each pixel in the adjacent frames is extracted using the calcOpticalFlowFarneback method in OpenCV. The motion vector and direction value of each pixel are obtained, and these motion information are mapped to the candidate area where the grasping point is located. The motion vector of each area is clustered and grouped. The points with the same motion direction are merged to further calculate the average speed of the area. The speed distribution of all motion vectors is averaged to determine the motion speed characteristics in the area. For areas with relatively uniform speed, they are marked as dynamically stable areas, while areas with large speed differences are marked as dynamically unstable areas. Based on the optical flow field, the spatial distribution characteristics of the motion speed and direction of the entire target object are analyzed. Finally, the overall motion speed of the area where the grasping point is located is marked to generate a motion speed information map.

[0092] S302: Based on the motion speed information graph and the motion direction distribution of the grasping point area, the directional consistency of the velocity vector in each area is analyzed, and the stability characteristics of the target motion are extracted by statistically analyzing the direction differences to obtain a motion stability analysis graph;

[0093] Based on the motion speed information graph and the motion direction distribution of the area where the grasping points are located, the motion vector direction of each area is analyzed point by point. MATLAB is used to visualize the motion vectors, and the motion direction of each point is represented by a vector arrow. The motion direction trend of each area is clearly marked, and the motion vector directions of all grasping points in each area are summarized. The direction consistency is judged by calculating the angular difference between each vector and the average direction of the area. The angular difference of all points in each area is calculated and counted point by point. If the proportion of points in the area that meet the direction consistency accounts for more than 80% of the total number of points, the area is marked as a motion stable area. Otherwise, it is marked as an unstable area, and a motion stability analysis diagram is obtained.

[0094] S303: Based on the motion stability analysis graph, sort the grasping points according to the stability level, adjust the sorting priority based on the motion speed and direction, and generate a dynamic sorting grasping point graph;

[0095] Based on the motion stability analysis diagram, the grasping points are sorted according to the stability level, and the stability level of the area where each grasping point is located is evaluated, and the stability level is divided into five levels from 1 to 5. First, the motion speed of each area is classified. The speed range is from 0 to 1m / s, corresponding to levels 1 to 5 respectively. The lower the speed, the higher the level. Then the consistency of direction change is evaluated. According to the statistical results of direction consistency in the previous paragraph, the area with a direction consistency ratio of more than 80% is judged as a high consistency area and given a higher stability level. The area with a direction consistency ratio between 50% and 80% is classified as medium stability. The area with a direction consistency ratio below 50% is regarded as a low consistency area. The stability level comprehensively considers speed and direction consistency. The area with both indicators at a high level is assigned the highest stability level. If one is high and the other is medium, it is assigned a medium-high level. And so on. Finally, the grasping points are sorted from high to low according to the stability level to generate a dynamic sorted grasping point diagram.

[0096] See also Figure 5 ,The specific steps of performing frequency domain analysis on the surface texture features of the ,region to which the grasping points of the dynamically sorted grasping point graph belong are as follows, ,to adjust the contact direction and force distribution of the grasping points, and ,generate surface optimized grasping point distribution information.

[0097] S401: Based on the dynamically sorted grab point map, extract the surface texture information of the area to which the grab point belongs, perform frequency domain characteristic analysis, separate the spatial distribution characteristics and directional information of the surface texture, and obtain a surface texture feature map;

[0098] Based on the dynamic sorting grab point map, the surface texture information of the area to which the grab point belongs is extracted. First, the texture information of the candidate area where the grab point is located is extracted. By performing Fourier transform on the image, the image data is converted from the spatial domain to the frequency domain. The separation characteristics of the high-frequency and low-frequency parts in the frequency domain are utilized to obtain the texture detail information of different frequency components respectively. The image is decomposed using Fourier analysis in MATLAB, and the high-frequency part is extracted as the detail texture feature, which describes the roughness and fine structure of the surface; at the same time, the low-frequency part is extracted to analyze the overall directional information of the surface, which describes the surface characteristics on a larger scale. The high-frequency and low-frequency characteristics are combined to reconstruct the surface texture feature map. These frequency components are inversely transformed, and the extracted frequency domain information is remapped back to the spatial domain to finally generate the surface texture feature map.

[0099] S402: Based on the surface texture feature map, the surface normal direction of the grasped area is obtained, the normal direction distribution is obtained by analyzing the phase consistency of the surface texture, the surface roughness value is measured and compared with a preset roughness threshold, and the area that meets the roughness requirement is screened to obtain the surface optimization feature map;

[0100] Based on the surface texture feature map, the surface normal direction of the grasping area is obtained, and phase consistency analysis is performed on each surface texture point. The phase value is obtained using MATLAB, and the phase difference between each adjacent point is compared. The change trend of the normal direction is identified by calculating the phase difference between adjacent points, thereby determining the normal direction distribution of the entire surface. For each surface area, its roughness characteristics are further measured. A needle-type surface roughness meter is used to perform surface measurement at multiple positions to obtain surface height change data. The roughness data of these measurement points are statistically analyzed to form overall roughness information. The roughness data is compared with the preset roughness threshold. If the roughness value is lower than the threshold, the area is marked as a suitable grasping area, and these screened areas are recorded to obtain a surface optimization feature map.

[0101] S403: Based on the surface optimization feature map, the contact direction and force distribution of the grasping point are optimized. By adjusting the angle between the normal direction and the contact surface of the manipulator gripper, the force range of the grasping point is evenly distributed, and the surface optimized grasping point distribution information is generated.

[0102] Based on the surface optimization feature map, the contact direction and force distribution of the grasping point are optimized. First, the normal direction of each grasping point is extracted, and these normal direction data are used to guide the adjustment of the robot arm gripper. The contact angle of the gripper is gradually adjusted through the robot arm's motion control system to ensure that the force direction of the gripper during grasping is as consistent as possible with the normal direction to reduce sliding caused by angle deviation. After determining the contact angle of the gripper, the force distribution of each grasping point is analyzed, and the force is simulated using finite element analysis to ensure that the force on the grasping point is uniform. The grasping points with uneven force distribution are adjusted by transferring part of the force to the surrounding points to achieve overall uniformity. Finally, the optimized grasping point distribution information is generated, and this information is used to guide the actual grasping operation.

[0103] See also Figure 6 Based on the surface optimization grasping point distribution information, combined with the robot arm path planning, dynamic path matching of the grasping points is performed, and the corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time. The specific steps for generating the dynamic grasping path of the robot arm are as follows:

[0104] S501: Analyze the spatial position of the grasping points based on the surface optimization grasping point distribution information, map the coordinates of the grasping points to the robot arm workspace, dynamically match the grasping points based on the robot arm motion trajectory, and generate grasping point path matching data;

[0105] First, the three-dimensional coordinates of all grasping points are extracted from the optimized grasping point distribution information and mapped to the robot's workspace coordinate system. By reading the robot's range of motion data and the end-effector's working area, the grasping points are confirmed to be within the robot's reach. Visualization tools are used to compare and annotate the grasping point distribution with the robot's working range. Grasping points within the reachable range are further analyzed for their priority in the robot's motion trajectory. The robot's end-effector's motion model is used to predict the optimal contact path and direction of the grasping points to reduce redundant paths in the robot's motion. A path planning algorithm is used to dynamically adjust the order of grasping points to ensure that the path of each grasping point does not conflict with the actual motion trajectory of the robot's end-effector. The path adjustment results are recorded as path matching data and imported into a simulation platform. The robot's dynamic grasping process is simulated to verify the compatibility of the grasping point path with the robot's working range, ultimately generating grasping point path matching data.

[0106] S502: Based on the grasping point path matching data, the spatial relationship between the grasping point and the center of gravity of the object is analyzed, the shape change of the object during the grasping action is predicted, the robot arm path planning is adjusted, and a shape change adjustment path diagram is obtained;

[0107] Analyze the spatial relationship between the grasping point and the center of gravity of the object. First, extract the three-dimensional coordinates of the grasping point and the center of gravity coordinates of the object. Use a depth camera or laser ranging device to obtain the position data of each grasping point and the center of gravity coordinates of the object as a whole. Perform vector difference calculation on the grasping point coordinates and the center of gravity coordinates to obtain the spatial displacement vector from each grasping point to the center of gravity of the object. Combined with the force vector of the grasping point measured by the force sensor, calculate the impact of each grasping point on the stability of the object's morphology. Based on the relationship between the comprehensive force and morphological changes of multiple grasping points, calculate the overall direction and amplitude of the object's morphological deviation. Use this result to evaluate the object's rotation tendency or position change during the grasping process.

[0108] To predict the morphological changes of objects during grasping, the formula is used:

[0109]

[0110] Calculate the displacement vector of the center of mass ;

[0111] in, Used to describe the displacement change of the object's center of mass during the grasping action. It is the total mass of an object, which is calculated by measuring the material density and volume of the object. The density is determined by weighing or experimental data, and the volume can be measured by 3D scanning. is the force vector acting on the grasping point, indicating the force acting on the The magnitude and direction of the force at each grasping point are measured in real time by the force sensor. The direction of the force vector indicates the direction in which the robot arm applies force. It is the spatial position vector of the grasping point, which represents the three-dimensional coordinate of each grasping point in space. It is measured by the visual system and the depth sensor. The coordinate is used to describe the geometric relationship between the grasping point and the center of gravity of the object. is a vector of adjustment coefficients used to modify the actual torque value of each gripping point, taking into account specific conditions such as the material properties, contact area, and depth of the gripping point. For example, when the gripping point is on a soft material or has a small contact area, the adjustment coefficient will increase to compensate for these effects. The adjustment coefficient is determined by experimentally analyzing the stability and force conditions of the gripping point. is the total number of grasping points, which represents the number of all grasping points identified on the object surface. Each grasping point is calibrated according to the physical characteristics and stability of the grasping area.

[0112] For example, the total mass of an object kg, the parameters of the three gripping points are as follows: Grasping point 1: m, N, , grab point 2: m, N, , grab point 3: m, N, , assuming that the calculation shows that: the moment of grip point 1 Grab point 2 torque Grab point 3 torque , the total torque is , the center of mass displacement is:

[0113]

[0114] The results show that the centroid Move up in the axial direction m, along Move up in the axial direction m, the displacement data is used to adjust the grasping path of the robot arm to ensure the morphological stability during the grasping action.

[0115] S503: Adjust the path diagram based on the morphological change, optimize the grasping direction and angle of the robot arm, output the adjusted grasping trajectory, and generate a dynamic grasping path for the robot arm.

[0116] First, the three-dimensional coordinates and surface normal direction of each grasping point in the adjusted path diagram are read. Combined with the working status of the robot arm's end effector, the grasping direction is analyzed point by point. Kinematic analysis tools are used to evaluate whether the grasping direction of the robot arm is consistent with the normal direction of the grasping point. If there is a deviation, the contact angle of the robot arm's gripper is adjusted to ensure that the force direction of the grasping point is as consistent as possible with the normal direction, thereby reducing the risk of grasping slip caused by angle deviation. Next, the force distribution of each grasping point is homogenized. The force range of the grasping point is simulated using a mechanical simulation tool, and the force is distributed in multiple neighborhoods around the grasping point. Areas with uneven force distribution are adjusted, and the force within the neighborhood is evenly distributed to ensure the overall force uniformity of the grasping point. Finally, the robot arm's grasping path is fully optimized, and the grasping point path and force distribution optimization results are synchronously updated to generate the final grasping direction and angle information. The adjusted grasping trajectory is output and the robot arm's dynamic grasping path is generated.

[0117] See also Figure 7 , a robotic arm grasping system based on image processing, the system includes:

[0118] The image processing module collects images of target objects using the visual sensor, adjusts brightness and contrast to balance the light distribution, screens the target area by analyzing layer texture changes, edge strength, and color contrast, marks potential grasping points, and generates a grasping point candidate area map;

[0119] The geometric characteristic analysis module extracts the three-dimensional geometric features of the candidate areas based on the grasping point candidate area map and projects them into a two-dimensional image. It determines the regional symmetry by analyzing the distribution deviation of the boundary points on both sides of the symmetry axis and detects the fluctuation amplitude of the boundary curvature. By selecting areas where both the symmetry deviation and the fluctuation amplitude meet the preset fluctuation threshold, it generates a geometric optimization grasping candidate area map.

[0120] The dynamic sorting module extracts a continuous image sequence of objects in the scene based on a geometrically optimized grasping candidate area map. It calculates the movement speed and direction of the grasping point area through optical flow analysis, determines the motion stability of the target object, prioritizes the grasping points, and generates a dynamically sorted grasping point map.

[0121] The path optimization module extracts the surface texture features of the area to which the grasping points belong based on the dynamic sorting grasping point map, measures the normal direction and roughness in the texture features, optimizes and adjusts the grasping points that exceed the roughness threshold, and dynamically matches the grasping points in combination with the robot arm motion trajectory. It adjusts the correspondence between the grasping point position and the robot arm motion trajectory in real time to generate a dynamic grasping path for the robot arm.

[0122] It should be understood that the term "and / or" as used herein simply describes a relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " as used herein generally indicates an "or" relationship between the associated objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0123] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0124] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0125] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present invention.

[0126] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0127] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0129] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0130] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0131] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A robotic arm grasping method based on image processing, characterized in that: The following steps are involved: The image of the target object that needs to be grasped by the robotic arm is collected through the visual sensor, and the collected target object image is preprocessed and multi-scale decomposition is performed to screen the area of ​​potential grasping points and generate a grasping point candidate area map; Extracting geometric features of the grasping point candidate region map, projecting the three-dimensional geometric shape of the candidate region into a two-dimensional image, analyzing and optimizing the boundary symmetry of the candidate region in the two-dimensional image, and generating a geometrically optimized grasping candidate region map; Performing dynamic characteristic analysis on the geometrically optimized grasping candidate area map to determine the motion stability of the target object, sorting the grasping points according to the motion stability of the target object, and generating a dynamically sorted grasping point map; Performing frequency domain analysis on surface texture features of the area to which the grasping points of the dynamically sorted grasping point graph belong, adjusting the contact direction and force distribution of the grasping points, and generating surface optimized grasping point distribution information; Based on the surface optimization grasping point distribution information, combined with the robot arm path planning, dynamic path matching is performed on the grasping points, and the corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time to generate a dynamic grasping path for the robot arm.

2. The robotic arm grasping method based on image processing according to claim 1, characterized in that: The grasping point candidate area map includes potential grasping points distributed in the target object surface texture uniform area, edge stable area, and color contrast area. The geometric optimization grasping candidate area map includes areas with boundary symmetry deviation within a preset range and areas with curvature change amplitude within a preset threshold range. The dynamic sorting grasping point map includes grasping points in the static area, grasping points whose movement speed matches the preset threshold, and grasping points whose movement direction changes match the preset range. The surface optimization grasping point distribution information includes grasping points whose normal direction is parallel to the target grasping path, grasping points whose surface roughness value matches the preset threshold, and grasping points whose contact direction is adjusted by force distribution. The dynamic grasping path of the robotic arm includes real-time matching of the grasping point position and the robotic arm movement trajectory, a path in which the grasping angle is aligned with the coordinate system, and trajectory adjustment of the grasping point position.

3. The robotic arm grasping method based on image processing according to claim 1, characterized in that: The specific steps for collecting images of the target object to be grasped by the robotic arm through the visual sensor, preprocessing and multi-scale decomposition of the collected target object images, screening the areas of potential grasping points, and generating the grasping point candidate area map are as follows: Based on the target object image captured by the visual sensor, the noise is removed by analyzing the pixel intensity distribution, the brightness is adjusted to balance the lighting area, and the processed target object image is obtained; Decomposing the processed target object image into multiple layers and detecting texture changes layer by layer, extracting color difference areas using pixel contrast values, accumulating gradient changes by obtaining edge strength, and generating a feature intensity map of the target object; Based on the feature intensity map of the target object, the curvature change trend is detected to verify the integrity of the boundary area, the directional features of the directional gradient distribution marked area are extracted, the potential grasping point area is screened, and the grasping point candidate area map is generated.

4. The robotic arm grasping method based on image processing according to claim 1, characterized in that: The specific steps of extracting the geometric features of the grab point candidate area map, projecting the three-dimensional geometric shape of the candidate area into a two-dimensional image, analyzing and optimizing the boundary symmetry of the candidate area in the two-dimensional image, and generating a geometrically optimized grab candidate area map are as follows: Extracting the three-dimensional surface morphology of the candidate area in the grasping point candidate area map, decomposing the area into a uniformly distributed dot matrix by gridding, reconstructing a two-dimensional representation of the three-dimensional morphology in a projection plane using height values ​​of the dot matrix, and generating a two-dimensional candidate area image; Based on the two-dimensional candidate region image, the position of the symmetry axis of each candidate region is detected, the number distribution deviation of the boundary points on both sides of the symmetry axis is counted, the fluctuation amplitude of the boundary curvature is calculated, and the region whose curvature fluctuation amplitude is within the fluctuation threshold is screened to obtain a symmetry optimization region map; Based on the symmetry optimization area map, the distribution density of the grasping points in the remaining area is analyzed, and by adjusting the spatial arrangement, the spatial position relationship of the grasping points in the area is optimized to generate a geometrically optimized grasping candidate area map.

5. The robotic arm grasping method based on image processing according to claim 4, characterized in that: For the statistical distribution deviation of the number of boundary points on both sides of the symmetry axis, the formula is used: Calculate the fluctuation amplitude of the boundary curvature ; in, It is The curvature value of the boundary point, is the mean curvature value, is the total number of boundary points.

6. The robotic arm grasping method based on image processing according to claim 1, characterized in that: The specific steps of performing dynamic characteristic analysis on the geometric optimization grasping candidate area map, determining the motion stability of the target object, sorting the grasping points according to the motion stability of the target object, and generating a dynamic sorting grasping point map are as follows: Based on the geometric optimization grasping candidate area map, the continuous image sequence of the object in the scene is used to extract optical flow information, the motion speed of the area where the grasping point is located is obtained, and the dynamic properties of the area are extracted by comparing the speed values ​​to obtain a motion speed information map; Based on the motion speed information graph, according to the motion direction distribution of the grasping point area, the directional consistency of the velocity vector in each area is analyzed, and the stability characteristics of the target motion are extracted by statistically analyzing the direction differences to obtain a motion stability analysis graph; Based on the motion stability analysis diagram, the grasping points are sorted according to the stability level, and the sorting priority is adjusted in combination with the motion speed and direction to generate a dynamic sorting grasping point diagram.

7. The robotic arm grasping method based on image processing according to claim 1, characterized in that: The specific steps of performing frequency domain analysis on the surface texture features of the area to which the grasping points of the dynamically sorted grasping point map belong, adjusting the contact direction and force distribution of the grasping points, and generating surface optimized grasping point distribution information are as follows: Based on the dynamically sorted grab point map, surface texture information of the area to which the grab points belong is extracted, frequency domain characteristic analysis is performed, spatial distribution characteristics and directional information of the surface texture are separated, and a surface texture feature map is obtained; Based on the surface texture feature map, the surface normal direction of the grasped area is obtained, the normal direction distribution is obtained by analyzing the phase consistency of the surface texture, the surface roughness value is measured and compared with a preset roughness threshold, the area matching the roughness requirement is screened, and the surface optimization feature map is obtained; Based on the surface optimization feature map, the contact direction and force distribution of the grasping point are optimized. By adjusting the angle between the normal direction and the contact surface of the robotic arm gripper, the force range of the grasping point is evenly distributed to generate surface optimized grasping point distribution information.

8. The robotic arm grasping method based on image processing according to claim 1, characterized in that: Based on the surface optimization grasping point distribution information, combined with the robot arm path planning, dynamic path matching is performed on the grasping points, and the corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time. The specific steps for generating the robot arm dynamic grasping path are as follows: Optimizing the distribution information of the grasping points on the surface, analyzing the spatial positions of the grasping points, mapping the coordinates of the grasping points to the workspace of the manipulator, dynamically matching the grasping points in combination with the motion trajectory of the manipulator, and generating grasping point path matching data; Based on the grasping point path matching data, the spatial relationship between the grasping point and the center of gravity of the object is analyzed, the morphological changes of the object during the grasping action are predicted, the path planning of the robot arm is adjusted, and a morphological change adjustment path diagram is obtained; Based on the morphological change, the path diagram is adjusted to optimize the grasping direction and angle of the robotic arm, and the adjusted grasping trajectory is output to generate a dynamic grasping path for the robotic arm.

9. The robotic arm grasping method based on image processing according to claim 8, characterized in that: To predict the morphological changes of objects during grasping, the formula is used: Calculate the displacement vector of the center of mass ; in, is the total mass of the object, Indicates the effect on The magnitude and direction of the force at each gripping point, is the spatial position vector of the grasping point, is the adjustment coefficient vector, is the total number of grab points.

10. The robotic arm grasping system based on image processing is characterized by: The image processing-based robotic arm grasping method according to any one of claims 1 to 9 is executed, wherein the system comprises: The image processing module collects images of target objects using the visual sensor, adjusts brightness and contrast to balance the light distribution, screens the target area by analyzing layer texture changes, edge strength, and color contrast, marks potential grasping points, and generates a grasping point candidate area map; The geometric characteristic analysis module extracts the three-dimensional geometric features of the candidate areas based on the grasping point candidate area map and projects them into a two-dimensional image. The module determines the symmetry of the area by analyzing the distribution deviation of the boundary points on both sides of the symmetry axis, detects the fluctuation amplitude of the boundary curvature, and generates a geometric optimization grasping candidate area map by selecting areas where both the symmetry deviation and the fluctuation amplitude meet the preset fluctuation threshold. The dynamic sorting module extracts a continuous image sequence of objects in the scene based on the geometric optimization grasping candidate area map, calculates the movement speed and direction of the grasping point area through optical flow analysis, determines the motion stability of the target object, sorts the priority of the grasping points, and generates a dynamic sorting grasping point map; The path optimization module extracts the surface texture features of the area to which the grasping points belong based on the dynamically sorted grasping point map, measures the normal direction and roughness in the texture features, optimizes and adjusts the grasping points that exceed the roughness threshold, and dynamically matches the grasping points in combination with the robot arm motion trajectory. The corresponding relationship between the grasping point position and the robot arm motion trajectory is adjusted in real time to generate a dynamic grasping path for the robot arm.

Images