A visual perception-based robotic arm grasping automatic fine-tuning method and system

Abstract

The application relates to the technical field of image processing, in particular to a mechanical arm grabbing automatic fine adjustment method and system based on visual perception. The method comprises the following steps: collecting a workpiece image; determining a first sub-feature based on vector distance after multi-scale analysis of image region division, constructing a second sub-feature based on a difference feature value of a gradient direction angle, and combining all two sub-features in a local region to obtain a first complex feature; taking a singular value condition of a matrix constructed based on the first complex feature as a weight to adjust a second complex feature, and then determining a complex factor; completing image processing by adjusting gray scales before and after filtering based on the complex factor; training a model after image processing, and then establishing a coordinate control mechanical arm grabbing by inputting an image. The application improves the mechanical arm grabbing precision.

Description

Technical Field

[0001] This application relates to the field of image processing technology, specifically to a method and system for automatic fine-tuning of robotic arm grasping based on visual perception. Background Technology

[0002] With the development of artificial intelligence, robotic arms are being used more and more widely in the field of logistics sorting, greatly promoting the level of automation in sorting. Robotic arms can autonomously complete sorting tasks, achieving efficient and rapid processing when handling large volumes of orders. This requires high grasping accuracy from the robotic arm. Therefore, in practical applications, achieving automatic fine-tuning of the robotic arm's grasping action is crucial.

[0003] In traditional logistics sorting, visual research typically relies solely on two-dimensional visual information to guide robotic arms in grasping tasks. While this improves the level of intelligence to some extent, it has significant limitations in adaptability to the placement and type of workpieces. To overcome these constraints, it is necessary to utilize binocular cameras to perform stereo imaging of the same target from different perspectives, thereby achieving high-precision target recognition and ranging, ultimately guiding the robotic arm to complete accurate and reliable sorting and grasping.

[0004] In the process of sorting logistics, vision-based robotic arms encounter scenarios with workpieces of different colors or workpieces arranged haphazardly. In such scenarios, the image data collected has high complexity in color and direction, and the boundaries between workpieces are relatively obvious. This can cause the image mean filtering algorithm to over-smooth the image during the filtering process, resulting in the destruction of the edge details of the workpiece pixels in the image. This reduces the accuracy of subsequent steps such as feature extraction and stereo matching, and ultimately leads to a decrease in the success rate of grasping when the robotic arm performs real-time fine-tuning. Summary of the Invention

[0005] To address the technical problem of inaccurate robotic arm grasping caused by low image precision, this application provides a vision-based automatic fine-tuning method and system for robotic arm grasping. The specific technical solution adopted is as follows:

[0006] In a first aspect, this application proposes a visual perception-based automatic fine-tuning method for robotic arm grasping, which includes the following steps:

[0007] Workpiece images are acquired using a binocular camera;

[0008] For a workpiece image, local regions are divided. For any pixel, its neighborhood is preset. Multiple color aggregation vectors are obtained through neighborhood multi-scale thresholding, and the first sub-feature of the pixel is determined based on the distance between them. The gradient direction angle value of the pixel is obtained by the gradient operator within the neighborhood. The gradient direction angle values ​​within the neighborhood are used to form a matrix and obtain a second-order difference matrix. The second sub-feature is obtained based on the proportion of eigenvalues ​​in the second-order difference matrix. The complex features of the pixel are obtained based on the first and second sub-features. The first complex feature of the local region is calculated based on the complex features of all pixels in the local region.

[0009] All the first complex features of the workpiece image are used to form a matrix, and the spatial features are determined by singular value decomposition; the first complex features are adjusted based on the spatial features to obtain the second complex features; and the complexity factor is obtained based on the first and second complex features.

[0010] The workpiece image is filtered based on mean filtering, with a complex factor as the weight. The adjusted gray value is obtained by adjusting the original gray level and the gray level after filtering, thus completing the processed image.

[0011] The target detection model is trained based on the processed images; the binocular images acquired by the binocular camera are used as the input to the target detection model to establish a point cloud coordinate system and control the robotic arm to grasp.

[0012] In the above-mentioned scheme, this application proposes to calculate the first complex feature of each region by analyzing the color and orientation information of pixels in each local region of the image, reflecting the color and orientation complexity of the region; and to reflect the importance of the first complex feature of each region in the overall image by analyzing the distribution degree of the first complex feature of all regions; when the robotic arm performs logistics sorting, the filtering algorithm is dynamically adjusted by the color and orientation features of different regions in the image, which effectively suppresses noise while significantly preserving edge and detail information, solving the problem that traditional mean filtering will over-smooth the image in logistics sorting scenarios with rich edge details, thereby improving the accuracy of the robotic arm in judging the position of the workpiece, enhancing the fine-tuning ability of the robotic arm in the approach and grasping stages, and finally achieving accurate and stable grasping of the workpiece.

[0013] In one embodiment, the method of obtaining multiple color aggregation vectors through neighborhood multi-scale thresholding and determining the first sub-feature of a pixel based on the distance between them is as follows:

[0014] For the neighborhood of each pixel, different multi-scale thresholds are set to obtain color aggregation vectors under different thresholds; the Euclidean distance between color aggregation vectors under adjacent thresholds is calculated, and the coefficient of variation of all Euclidean distances is calculated, normalized, and used as the first sub-feature of the pixel.

[0015] In one embodiment, the method of obtaining the gradient direction angle value of a pixel in the neighborhood using a gradient operator, constructing a matrix from the gradient direction angle values ​​in the neighborhood, and obtaining a second-order difference matrix is ​​as follows:

[0016] For each neighborhood, the gradient vector of all pixels in the neighborhood is calculated using the gradient operator, and the gradient direction angle value is obtained. The gradient direction angle values ​​of all pixels in the neighborhood are used to form an angle matrix. The second-order difference matrix of the angle matrix is ​​obtained by the second-order difference algorithm. When calculating the difference value, the absolute value of the difference between the angles is recorded as the first angle, the difference between 2π and the first angle is recorded as the second angle, and the minimum value of the first angle and the second angle is used as the difference value.

[0017] In one embodiment, the method for obtaining the second sub-feature based on the eigenvalue ratio of the second-order difference matrix is ​​as follows:

[0018] For a second-order difference matrix, the PCA algorithm is used to perform eigenvalue decomposition, and the ratio of the largest eigenvalue to the sum of all eigenvalues ​​is taken as the second sub-feature of the pixel.

[0019] In one embodiment, the complex feature is positively correlated with the first sub-feature and the second sub-feature, respectively.

[0020] In one embodiment, the first complex feature of the local region is the coefficient of variation of the complex features of all pixels within the local region.

[0021] In one embodiment, the method of constructing a matrix from all the first complex features of the workpiece image and determining spatial features through singular value decomposition; and adjusting the first complex features based on the spatial features to obtain the second complex features is as follows:

[0022] Singular value decomposition is performed on the matrix constructed from all the first complex features of the workpiece image to obtain singular values. The condition number of the matrix is ​​calculated using the maximum and minimum singular values ​​as the spatial features of the workpiece image.

[0023] After normalizing the first complex feature by its maximum and minimum values, the product of the normalized first complex feature and the spatial feature is taken as the second complex feature of the local region.

[0024] In one embodiment, the complexity factor is positively correlated with the first complexity feature and the second complexity feature, respectively.

[0025] In one embodiment, the adjusted grayscale value is obtained by weighting the grayscale value before and after filtering with a complexity factor. The larger the complexity factor, the greater the influence of the grayscale value before filtering; the smaller the complexity factor, the greater the influence of the grayscale value after filtering.

[0026] On the other hand, this application also provides a vision-based robotic arm grasping automatic fine-tuning system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any of the above-described vision-based robotic arm grasping automatic fine-tuning methods.

[0027] The beneficial effects of this application are as follows:

[0028] This application proposes to calculate the first complex feature of each region by analyzing the color and orientation information of pixels in each local region of an image, reflecting the color and orientation complexity of that region; and to reflect the importance of the first complex feature of each region in the overall image by analyzing the distribution of the first complex feature of all regions. When a robotic arm performs logistics sorting, the filtering algorithm is dynamically adjusted by the color and orientation features of different regions in the image. While effectively suppressing noise, it significantly preserves edge and detail information, solving the problem that traditional mean filtering will over-smooth the image in logistics sorting scenarios with rich edge details. This improves the accuracy of the robotic arm in judging the position of the workpiece, enhances the fine-tuning capability of the robotic arm in the approach and grasping stages, and ultimately achieves precise and stable grasping of the workpiece. Attached Figure Description

[0029] To more clearly illustrate the technical solutions and advantages in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 This is a flowchart of a vision-based robotic arm grasping automatic fine-tuning method provided in one embodiment of this application. Detailed Implementation

[0031] To further illustrate the technical means and effects adopted by this application to achieve the intended inventive purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a vision-perception-based robotic arm grasping automatic fine-tuning method and system proposed in this application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0032] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0033] An embodiment of a vision-based robotic arm grasping automatic fine-tuning method and system:

[0034] The following description, in conjunction with the accompanying drawings, details a specific scheme for an automatic fine-tuning method for gripping robotic arms based on visual perception, as provided in this application.

[0035] Please see Figure 1 The diagram illustrates a flowchart of a vision-based robotic arm grasping automatic fine-tuning method according to an embodiment of this application. The method includes the following steps:

[0036] Step S001: Acquire workpiece images using a binocular camera.

[0037] A binocular camera is placed at the end of the robotic arm to capture binocular images of the workpiece to be grasped in real time; in this embodiment, the number of frames captured is 25.

[0038] During the logistics sorting process, the robotic arm acquires consecutive frames of workpiece images while performing a grasping task. During acquisition, the robotic arm's base is installed using anchor bolts or welding to prevent wobbling during movement that could affect subsequent processing. For example: ① Motion blurring occurs in the images, causing loss of workpiece edge details; ② Opposite displacement between the workpiece and background in consecutive frames causes image jitter; ③ Changes in the robotic arm's base coordinate system lead to offsets between the camera coordinate system, the robotic arm's base coordinate system, and the end effector joint coordinate system, causing stereo matching failure between the binocular cameras and resulting in coordinate inconsistencies. Ultimately, this causes the robotic arm's grasping fine-tuning to fail, preventing it from completing the logistics sorting task.

[0039] At this point, the workpiece image was acquired using a binocular camera.

[0040] Step S002: After performing multi-scale analysis on the image region segmentation, determine the first sub-feature by vector distance, construct the second sub-feature based on the differential feature value of the gradient direction angle, and obtain the first complex feature by combining all two sub-features in the local region.

[0041] During the logistics sorting process, workpieces that need to be sorted at different times may have different colors or be placed in a disorderly manner. At this time, the color and orientation features of workpiece pixels at different positions in the same image will be different, indicating that the edge detail features at different positions are different. If the same filtering weight is used for processing, it will increase the loss of edge detail features in the image.

[0042] Therefore, this application divides the image into local regions, taking the image as input and setting the grid size, with a value range of [64, 256]. In this embodiment, the value is 64. The image is then segmented into multiple regular local regions using NumPy functions; grid division using NumPy functions is a well-known technique in the art, and the specific calculation method will not be elaborated further.

[0043] Setting the neighborhood of each pixel: Taking any pixel in any region of the image as the center, take a square window of size n×n. The area covered by this window is the neighborhood of that pixel. The missing values ​​in the neighborhood of edge pixels are filled in using a linear interpolation algorithm. In this embodiment, n is 15. The linear interpolation algorithm is a well-known technology in the field, and the specific calculation method will not be described in detail.

[0044] The greater the color difference of the workpiece, the more obvious the color features may be, leading to a more discrete spatial distribution of color features in the image. Under a fixed threshold, the color aggregation vector struggles to reliably distinguish true color clustering patterns: an overly lenient threshold may misclassify sporadically distributed heterochromatic pixels as aggregation features, while an overly strict threshold may ignore truly existing small-sized color clustering regions. Therefore, for the neighborhood of each pixel, a multi-scale threshold for the color aggregation vector is set. This threshold must satisfy the condition of being less than 1% of the image's side length. In this embodiment, the multi-scale threshold values ​​are {3, 4, 5, 6}. For each threshold, the color aggregation vector of the neighborhood is obtained. The calculation method for the color aggregation vector is a well-known technique and will not be described in detail in this application.

[0045] For each pixel, the Euclidean distance between the color aggregation vectors corresponding to two adjacent thresholds in its neighborhood is calculated. The coefficient of variation of all Euclidean distance values ​​is calculated and normalized using the min-max algorithm, and then recorded as the first sub-feature of the pixel. The greater the color complexity of the workpiece, the more complex the color features in the corresponding neighborhood, and the weaker the color aggregation. Using the first sub-feature can highlight the color aggregation of all pixel gradients in the neighborhood. The normalized mapping range is [0,1].

[0046] Since logistics workpieces typically have obvious linear edge features, haphazard placement will cause these linear edges to be distributed disorderly in the image, resulting in significant differences in the gradient directions of all pixels within the neighborhood of each pixel. Therefore, for each pixel's neighborhood, the gradient direction angle values ​​of all pixels in the neighborhood are calculated. In this embodiment, the gradient direction calculation method uses the Sobel gradient operator. For each pixel in the neighborhood, the gradient direction angle values ​​corresponding to each pixel are combined into an n×n matrix according to the pixel's position in the neighborhood. The matrix is ​​used as input and processed using a second-order difference algorithm. During the processing, since the gradient direction angle is periodic, it is necessary to process all gradient direction angle values. Therefore, in this embodiment, the difference value is min(|angle value 1 - angle value 2|, 2...). -|angle value 1 - angle value 2|); where min() is the minimum value function; thus, the second-order difference matrix is ​​output. The Sobel edge detection operator and the second-order difference algorithm are well-known techniques and will not be described in detail in this application. n is the side length of the square window, and also represents the side length of the neighborhood.

[0047] Furthermore, the PCA algorithm is used to perform eigenvalue decomposition on the second-order difference matrix, and the three eigenvalues ​​in the second-order difference matrix are calculated. The proportion of the largest eigenvalue to the sum of all eigenvalues ​​is taken as the second sub-feature of the pixel. The second-order difference matrix can reflect the degree of change of the gradient direction angle value of all pixels in the neighborhood. The greater the degree of change of the gradient direction angle value, the more obvious the linear edge features in the neighborhood. In the logistics sorting scenario, the more disordered the placement of workpieces, the more complex the distribution of linear edge features in the corresponding image area, and the more discrete the gradient direction of all pixels. Using the second-order difference matrix can highlight the degree of dispersion of the gradient direction of all pixels in the neighborhood.

[0048] The complex features of each pixel are obtained based on the first and second sub-features of each pixel.

[0049] The complex features are positively correlated with the first sub-feature and the second sub-feature, respectively.

[0050] It should be noted that positive correlation means that when one variable increases, the other variable also increases, and the two variables change in the same direction. When one variable changes from large to small or from small to large, the other variable also changes from large to small or from small to large. The specific relationship is determined by the actual application, and this application does not impose any special restrictions.

[0051] Preferably, in this embodiment, the expression for the complex features of a pixel is:

[0052] , This represents the first sub-feature of the i-th pixel. This represents the second sub-feature of the i-th pixel. Represents the complex features of the i-th pixel. This represents the normalization function; in this embodiment, the maximum and minimum value normalization function is used.

[0053] Preferably, in another embodiment, the expression for the complex features of a pixel is:

[0054] , This represents the first sub-feature of the i-th pixel. This represents the second sub-feature of the i-th pixel. Represents the complex features of the i-th pixel. This represents the normalization function.

[0055] Let represent the first sub-feature of the i-th pixel, which can characterize the color complexity within the neighborhood. The larger the first sub-feature value, the greater the color difference between different workpieces at corresponding locations in a logistics sorting scenario; this indicates that the larger the first sub-feature value, the greater the color complexity of the neighborhood. The larger; Let represent the second sub-feature of the i-th pixel. This second sub-feature characterizes the distribution of gradient directions of all pixels in the neighborhood. The larger the second sub-feature, the more drastic the change in gradient directions of all pixels in the neighborhood, and the greater the complexity of the directional distribution in the neighborhood. The larger.

[0056] Furthermore, if the proportion of workpiece pixels in a local area is high or the distribution is relatively scattered, it indicates that there are many detailed features in that local area. In order to characterize the complexity of the detailed features of each local area, for each local area, the coefficient of variation of the complex features of all pixels in the local area is calculated, and this value is used as the first complex feature of the local area.

[0057] At this point, the first complex feature of each local region has been obtained.

[0058] Step S003: Based on the singular value conditions of the matrix constructed from the first complex feature, the second complex feature is adjusted as a weight to determine the complex factor.

[0059] Because the degree of clutter varies between different local regions, special cases may occur. For example, a local region may have a low first complexity feature, but other local regions in the same image may have even lower first complexity features. In this case, the first complexity feature of the local region may actually be higher than that of the other local regions. Therefore, it is necessary to analyze the relative relationships between local regions to determine the degree of complexity.

[0060] Based on the positional relationship of the center points of all local regions in an image, the first complex feature corresponding to each local region is used as an element in a matrix. Each workpiece image is then composed of an M×M matrix, where M×M represents the number of local regions. This matrix is ​​used as input, and the singular value decomposition (SVD) algorithm is used to calculate the three singular values ​​of the matrix. The condition number of the matrix is ​​calculated based on the maximum and minimum singular values, serving as a spatial feature of the image. The M×M matrix can characterize the distribution of color direction features of all pixels in the entire image. The condition number calculated based on the maximum and minimum singular values ​​in the SVD algorithm reflects the stability of all elements in the M×M matrix. The more unstable the elements, the more uneven the distribution of color direction features of all pixels in the entire image. In logistics sorting scenarios, the color and disorder of workpieces at different locations vary. The condition number is a well-known technique in singular value matrix decomposition and will not be elaborated upon here.

[0061] For the first complex feature, the min-max normalization is applied, and the product of the normalized first complex feature and the spatial feature of each local region is taken as the second complex feature of each local region.

[0062] The complexity factor of each local region is obtained by combining the first and second complex features.

[0063] The complexity factor is positively correlated with the first complexity feature and the second complexity feature, respectively.

[0064] Preferably, in this embodiment, the expression for the complexity factor is:

[0065] , This represents the first complex feature of the a-th local region. This represents the second complex feature of the a-th local region. Represents the normalization function. This represents the complexity factor of the a-th local region.

[0066] The greater the first complexity feature, the greater the color direction complexity within the local area, indicating that during the logistics sorting process, the color differences between different workpieces at corresponding positions in the local area are significant and their placement is relatively disordered. The greater the second complexity feature, the greater the color direction complexity of the local area compared to other local areas in the same image, indicating that the color differences of workpiece pixels in the local area are greater than those in other local areas. In this case, the filtering degree for the local area should be reduced to retain more detailed features within the local area.

[0067] At this point, the complexity factors for each local region have been obtained.

[0068] Step S004: Image processing is completed by adjusting the grayscale before and after filtering based on complex factors.

[0069] The complexity factor of each local region is obtained through the above steps. The image and the complexity factor of each local region are used as input. The size of the mean filtering algorithm is set, and its value range is {3, 5, 7}. In this embodiment, the empirical value is 3. The mean filtering algorithm is used to perform mean filtering on the image. For each pixel of each local region in an image, the mean filtering is first performed on the pixel through the pixels in the neighboring area, and then the gray value after mean filtering is adjusted by the complexity factor.

[0070] Using a complexity factor as the weight, the gray value of the pixel itself and the gray value after mean filtering are weighted and summed. The larger the complexity factor, the more detailed features are needed, so more gray values ​​of the pixel itself are retained, the filtering weight of the pixel is reduced, and more detailed features are retained. The smaller the complexity factor, the lower the complexity of the local region, and the filtering result is more similar to the output result of the original mean filtering, so a large-scale filtering is required.

[0071] Based on this, the adjusted grayscale value of each pixel is obtained.

[0072] Preferably, in this embodiment, the expression for adjusting the grayscale value is:

[0073] , The complexity factor represents the a-th local region. This represents the original grayscale value of the k-th pixel in the a-th local region. This represents the grayscale value of the k-th pixel in the a-th local region after mean filtering. This represents the adjusted grayscale value of the k-th pixel in the a-th local region.

[0074] This completes the filtering of all pixels in each workpiece image, achieving adaptive filtering of the workpiece image and obtaining the processed image.

[0075] At this point, the processed image of each workpiece has been obtained.

[0076] Step S005: Train the model based on the processed image, and then establish coordinates through the input image to control the robotic arm to grasp.

[0077] The processed images are labeled using the Labelme annotation tool, with the label shape set to rectangles. Each image is output as a label file. The processed images and their corresponding labels are taken as input. The loss function is set to distributed focal loss, the optimizer is set to SGD, and the number of iterations is set to 500. The YOLOv8 model is used to perform object detection on the processed images to obtain a trained object detection model.

[0078] When the robotic arm performs a grasping task, a stereo camera is used to acquire consecutive frames of stereo images of the target workpiece. Each frame of the stereo image is used as input, and maximum and minimum cost parameters are set. In this embodiment, empirical values ​​of 64 and 0 are used, respectively. The BT algorithm is used to perform stereo matching on the stereo images of the target workpiece to generate a disparity map. Based on the disparity map and the intrinsic and extrinsic parameters of the stereo camera, the target workpiece is reconstructed in three dimensions to obtain a three-dimensional point cloud. The point cloud is then transformed from the camera coordinate system to the robotic arm base coordinate system using a hand-eye calibration method to obtain the three-dimensional spatial position coordinates of the target workpiece in the robotic arm base coordinate system. The BT algorithm, the method for acquiring the intrinsic and extrinsic parameters of the stereo camera, the three-dimensional reconstruction, and the hand-eye calibration method are well-known technologies in the field, and the specific calculation methods will not be described in detail.

[0079] Furthermore, the three-dimensional spatial position sequence of the target workpiece in the robot arm's base coordinate system, calculated from consecutive frames, is used as input. The workpiece's position and velocity are used as state variables. The Kalman filter algorithm is used to perform three-dimensional tracking and prediction of its motion to obtain the predicted position. When the robot arm performs a grasping task, the time for the robot arm to grasp a workpiece once is obtained from the log. Calculate the current position interval of the workpiece based on the desired gripping point position. The positional deviation between the predicted position and the desired grasping point position is calculated; a control gain is set with a range of [0,1], and an empirical value of 0.5 is used in this embodiment; the inverse of the Jacobian matrix is ​​the unit Jacobian matrix, and the positional basis visual servo (PBVS) algorithm is used to convert the positional deviation into a three-dimensional motion compensation amount for the robotic arm end effector. The Kalman filter algorithm and the positional basis visual servo (PBVS) algorithm are well-known technologies in this field, and the specific calculation methods will not be elaborated further.

[0080] The three-dimensional motion compensation command is sent to the controller of the robotic arm in real time. After receiving the command, the controller converts the three-dimensional motion compensation into a real-time motion command for the end effector. The current motion is adjusted through the real-time motion command to achieve automatic fine-tuning of the robotic arm's grasping.

[0081] Based on the same inventive concept as the above method, this embodiment of the invention also provides a vision-based robotic arm grasping automatic fine-tuning system, including a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of any one of the above-described vision-based robotic arm grasping automatic fine-tuning methods.

[0082] It should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

[0083] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for automatic fine-tuning of a robotic arm grasping based on vision perception, characterized in that, The method includes the following steps: Workpiece images are acquired using a binocular camera; For a workpiece image, local regions are divided. For any pixel, its neighborhood is preset. Multiple color aggregation vectors are obtained through neighborhood multi-scale thresholding, and the first sub-feature of the pixel is determined based on the distance between them. The gradient direction angle value of the pixel is obtained by the gradient operator within the neighborhood. The gradient direction angle values ​​within the neighborhood are used to form a matrix and obtain a second-order difference matrix. The second sub-feature is obtained based on the proportion of eigenvalues ​​in the second-order difference matrix. The complex features of the pixel are obtained based on the first and second sub-features. The first complex feature of the local region is calculated based on the complex features of all pixels in the local region. All the first complex features of the workpiece image are used to form a matrix, and the spatial features are determined by singular value decomposition; the first complex features are adjusted based on the spatial features to obtain the second complex features; and the complexity factor is obtained based on the first complex features and the second complex features. The workpiece image is filtered based on mean filtering, with a complex factor as the weight. The adjusted gray value is obtained by adjusting the original gray level and the gray level after filtering, thus completing the processed image. The target detection model is trained based on the processed images; the binocular images acquired by the binocular camera are used as the input to the target detection model to establish a point cloud coordinate system and control the robotic arm to grasp.

2. The method for automatic fine-tuning of a robotic arm grasping based on vision perception as described in claim 1, characterized in that, The method for obtaining multiple color aggregation vectors through neighborhood multi-scale thresholding and determining the first sub-feature of a pixel based on the distance between them is as follows: For the neighborhood of each pixel, different multi-scale thresholds are set to obtain color aggregation vectors under different thresholds; the Euclidean distance between color aggregation vectors under adjacent thresholds is calculated, and the coefficient of variation of all Euclidean distances is calculated and normalized as the first sub-feature of the pixel.

3. The method for automatic fine-tuning of a robotic arm grasping based on vision perception as described in claim 1, characterized in that, The method for obtaining the gradient direction angle value of a pixel in the neighborhood using a gradient operator, constructing a matrix from the gradient direction angle values ​​in the neighborhood, and obtaining a second-order difference matrix is ​​as follows: For each neighborhood, the gradient vector of all pixels in the neighborhood is calculated using the gradient operator, and the gradient direction angle value is obtained. The gradient direction angle values ​​of all pixels in the neighborhood are used to form an angle matrix. The second difference matrix of the angle matrix is ​​obtained by the second difference algorithm. When calculating the difference value, the absolute value of the difference between the angles is recorded as the first angle, the difference between 2π and the first angle is recorded as the second angle, and the minimum value of the first angle and the second angle is taken as the difference value.

4. The method for automatic fine-tuning of a robotic arm grasping based on vision perception as described in claim 1, characterized in that, The method for obtaining the second sub-feature based on the proportion of eigenvalues ​​of the second-order difference matrix is ​​as follows: For a second-order difference matrix, the PCA algorithm is used to perform eigenvalue decomposition, and the ratio of the largest eigenvalue to the sum of all eigenvalues ​​is taken as the second sub-feature of the pixel.

5. The automatic fine-tuning method for robotic arm grasping based on vision perception as described in claim 1, characterized in that, The complex features are positively correlated with the first sub-feature and the second sub-feature, respectively.

6. The automatic fine-tuning method for robotic arm grasping based on vision perception as described in claim 1, characterized in that, The first complex feature of the local region is the coefficient of variation of the complex features of all pixels within the local region.

7. The method for automatic fine-tuning of a robotic arm grasping based on vision perception as described in claim 1, characterized in that, The method of constructing a matrix from all the first complex features of the workpiece image and determining spatial features through singular value decomposition; and adjusting the first complex features based on the spatial features to obtain the second complex features is as follows: Singular value decomposition is performed on the matrix constructed from all the first complex features of the workpiece image to obtain singular values. The condition number of the matrix is ​​calculated using the maximum and minimum singular values ​​as the spatial features of the workpiece image. After normalizing the first complex feature by its maximum and minimum values, the product of the normalized first complex feature and the spatial feature is taken as the second complex feature of the local region.

8. The automatic fine-tuning method for robotic arm grasping based on vision perception as described in claim 1, characterized in that, The complexity factor is positively correlated with the first complexity feature and the second complexity feature, respectively.

9. The automatic fine-tuning method for robotic arm grasping based on vision perception as described in claim 1, characterized in that, The adjusted grayscale value is obtained by weighting the grayscale value before and after filtering with a complexity factor. The larger the complexity factor, the greater the influence of the grayscale value before filtering; the smaller the complexity factor, the greater the influence of the grayscale value after filtering.

10. A vision-based robotic arm grasping automatic fine-tuning system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the visual perception-based robotic arm grasping automatic fine-tuning method as described in any one of claims 1-9.

Images