Mechanical arm grabbing intelligent control method and system based on pose online correction

Abstract

The invention discloses a mechanical arm grabbing intelligent control method and system based on pose online correction, and relates to the field of mechanical arm intelligent control. The method comprises the steps that a mechanical characteristic matrix is detected through a mechanical sensor at the tail end of a mechanical arm, and drift error prediction is conducted; a binocular vision sensor on the mechanical arm is used for collecting binocular images, registration analysis is carried out, and registration drift errors and registration drift credibility are obtained; performing verification to obtain drift consistency parameters, and performing labeling in combination with registration drift credibility to obtain a drift prediction training data set; and the identification information consistency is analyzed, identification stability parameters are obtained, prediction drift errors and registration drift errors are fused, a processed drift prediction training data set is obtained, and mechanical arm correction control and drift error predictor updating training are carried out. The problems that in the prior art, error monitoring of detection equipment is not accurate, errors are too large in the correction process, the correction effect is poor, and the detection equipment cannot adapt to the complex industrial environment are solved.

Description

Technical Field

[0001] This invention relates to the field of intelligent control of robotic arms, specifically to an intelligent control method and system for robotic arm grasping based on online pose correction. Background Technology

[0002] With the continuous improvement of intelligent manufacturing and production demands, industrial robotic arms are increasingly widely used in grasping operations such as assembly, sorting, and loading / unloading. However, during long-term, continuous operation, the absolute positioning accuracy of the robotic arm system can drift due to various factors.

[0003] Currently, error correction in robotic arms primarily relies on online correction using visual servoing or visual guidance. This involves real-time observation of the target or reference point via a camera to calculate and compensate for pose deviations. However, in complex industrial scenarios, the reliability of visual perception is highly dependent on ambient lighting and unobstructed conditions. The presence of reflections or temporary occlusion of the target can easily introduce significant noise into the visual information. Furthermore, single visual observations may not be accurate enough to detect errors due to image processing algorithm errors, hindering the adaptive adjustment of control strategies. Summary of the Invention

[0004] This application provides a robotic arm grasping intelligent control method and system based on online pose correction, which addresses the problem in the prior art that inaccurate error monitoring of detection equipment leads to excessive errors during the correction process, resulting in poor correction effect and inability to adapt to complex industrial environments.

[0005] In view of the above problems, this application provides a robotic arm grasping intelligent control method and system based on online pose correction.

[0006] In a first aspect, this application provides a robotic arm grasping intelligent control method based on online pose correction, the method comprising: By using a mechanical sensor configured at the end of the robotic arm, a mechanical feature matrix is ​​detected during the grasping process. A drift error predictor is then used to predict the drift error, thus obtaining the predicted drift error. By using a binocular vision sensor configured on the robotic arm, binocular images are acquired, and registration analysis is performed to obtain the registration drift error and registration drift reliability. The predicted drift error and registration drift error are verified to obtain the drift consistency parameter. Combined with the registration drift confidence, the mechanical feature matrix, the predicted drift error and the registration drift error are labeled to obtain the drift prediction training data set. The consistency of the identification information between the drift prediction training data set and the cumulative drift prediction training data set is analyzed to obtain the identification stability parameter. The prediction drift error and the registration drift error are fused to obtain the processed drift prediction training data set, which is then used for robot arm correction control and drift error predictor update training.

[0007] Secondly, this invention provides an intelligent control system for robotic arm grasping based on online pose correction, the system comprising: The drift error prediction module is used to detect the mechanical feature matrix during the grasping process by using a mechanical sensor configured at the end of the robotic arm, and then use a drift error predictor to predict the drift error. The registration analysis module is used to acquire binocular images through a binocular vision sensor configured on the robotic arm, perform registration analysis, and obtain the registration drift error and registration drift confidence. The registration confidence annotation module is used to verify the prediction drift error and the registration drift error, obtain the drift consistency parameter, and, in combination with the registration drift confidence, annotate the mechanical feature matrix, the prediction drift error and the registration drift error to obtain the drift prediction training data set. The calibration update module is used to analyze the consistency of the identification information between the drift prediction training data set and the cumulative drift prediction training data set, obtain the identification stability parameter, fuse the prediction drift error and the registration drift error to obtain the processed drift prediction training data set, and perform update training for the robotic arm calibration control and the drift error predictor.

[0008] One or more technical solutions provided in this application have at least the following technical effects or advantages: This application firstly, by using a mechanical sensor configured at the end of a robotic arm, a mechanical feature matrix is ​​detected during the grasping process. The dynamic mechanical information of contact with the object during grasping is converted into drift error prediction for correction. Simultaneously, a drift error predictor is constructed to predict the drift error, thereby improving the stability and accuracy of the prediction. Secondly, by using a binocular vision sensor configured on the robotic arm, binocular images are acquired. Visual observation of the robotic arm in its reset state after grasping ensures the consistency of the measurement reference. This improves registration analysis, calculates the drift error of the fixed reference target, and evaluates the reliability of the visual measurement to obtain the registration drift error and registration drift reliability, thus improving the reliability of error correction under different environments.

[0009] Next, the prediction drift error and registration drift error are validated. Through online cross-validation of force prediction and visual observation, and data quality fusion evaluation, the consistency parameter of the two error vectors is calculated, effectively identifying sensor anomalies or interference. Furthermore, combining the inherent reliability of vision, a fusion confidence score is generated, and the mechanical feature matrix, prediction drift error, and registration drift error are labeled to obtain the drift prediction training data set. This data is used for model updates, providing a high-quality data foundation for subsequent model iteration and optimization. Finally, the consistency of the labeling information between the drift prediction training data set and the cumulative drift prediction training data set is analyzed to obtain the labeling stability parameter. Anomalies in the current state are identified over time. Based on this stability parameter, weights are dynamically adjusted for adaptive weighted fusion to derive the fusion drift deviation, which is then used to correct and control the robotic arm's grasping action, achieving precise compensation. Once a sufficient amount of high-quality accumulated data is available, incremental learning is performed, enabling the drift error predictor to continuously optimize using operational data and adapt to long-term changes. Attached Figure Description

[0010] Figure 1 This is a flowchart illustrating the intelligent control method for robotic arm grasping based on online pose correction proposed in this application. Figure 2 This is a schematic diagram of the intelligent control system for robotic arm grasping based on online pose correction, as described in this application.

[0011] In the attached diagram, the components represented by each number are as follows: The module includes a drift error prediction module 11, a registration analysis module 12, a registration confidence labeling module 13, and a correction update module 14. Detailed Implementation

[0012] This application provides an intelligent control method for robotic arm grasping based on online pose correction, which specifically solves the problem in the prior art that the error monitoring of the detection equipment is inaccurate, resulting in excessive error during the correction process, poor correction effect, and inability to adapt to complex industrial environments.

[0013] The present invention will now be described in detail with reference to the accompanying drawings.

[0014] Example 1, as Figure 1 As shown, this application provides an intelligent control method for robotic arm grasping based on online pose correction, the method comprising: S10: During the grasping process, the mechanical feature matrix is ​​detected by the mechanical sensor configured at the end of the robotic arm. The drift error is predicted by the drift error predictor. In this embodiment, the mechanical feature matrix is ​​a matrix used to describe mechanical characteristics such as stiffness, mass, and stability in mechanical problem analysis; the drift error predictor is a neural network model trained using machine learning algorithms and supervised learning training methods, which is then used to predict drift errors.

[0015] Specifically, firstly, a corresponding mechanical sensor is configured at the end of the robotic arm, and the mechanical characteristics in each direction are monitored by the mechanical sensor during each grasp. Since the robotic arm may undergo structural changes when subjected to load, the larger the magnitude of the structural change error, the more likely a drift error predictor is to be constructed to predict the drift error, which includes the error direction and distance.

[0016] Step S10 in the method provided in this application embodiment includes: By using a mechanical sensor configured at the end of the robotic arm, a mechanical feature matrix is ​​obtained during the grasping process. The mechanical feature matrix includes mechanical feature values ​​in multiple directions. The mechanical feature matrix is ​​input into the drift error predictor, and the predicted drift error is output.

[0017] In this embodiment, the mechanical feature value is the specific force or torque value in each direction obtained by the mechanical sensor at a single sampling moment or after preliminary processing; the predicted drift error is the estimate of the systematic deviation of the current robotic arm end pose from the theoretical expected pose by the drift error predictor based on the input mechanical feature matrix and through complex internal calculations.

[0018] Specifically, firstly, force sensors are configured at the corresponding positions at the end of the robotic arm. For example, several force sensors can be configured at the top and bottom of the end of the robotic arm. Within a specific time window triggered by the grasping action, the readings of the force sensors are continuously collected at a fixed frequency as the mechanical characteristic values ​​generated during the grasping process. The mechanical characteristic value of each reading includes the magnitude of the force and torque in the corresponding direction. If the number of sampling points is N, the time series data is arranged into a T×N mechanical characteristic matrix, where T is the time step.

[0019] Secondly, after each grasping action is completed and the corresponding mechanical feature matrix is ​​generated, the mechanical feature matrix is ​​used as input and passed to the drift error predictor model. The model undergoes multiple nonlinear transformations and finally outputs a predicted drift error.

[0020] In step S10 of the method provided in this application embodiment, the training step of the drift error predictor includes: Based on the monitoring data of robot arm drift error over a historical period, a set of sample mechanical feature matrices was collected, and the scale and direction of the drift deviation sent by the robot arm under different sample mechanical feature matrices were collected and labeled to obtain a set of sample drift errors. A drift error predictor is constructed based on machine learning, wherein the drift error predictor includes multiple sets of initial weights and biases; The drift error predictor is trained and optimized using the set of sample mechanical feature matrices as training input and the set of sample drift errors as training supervision labels until the test converges, thus completing the training.

[0021] In this embodiment, the robotic arm drift error monitoring data is the drift error monitoring data generated during the robotic arm grasping process within a certain historical period; the scale and direction of the drift deviation are the specific values ​​of the pose error measured by the monitoring device at the end of the robotic arm during a grasping operation; the annotation is to mark different result data through an annotation tool, and then use them as sample data for supervised learning.

[0022] First, within a certain historical period, a robotic arm drift error monitoring data acquisition system is used to collect robotic arm drift error monitoring data. During each grasping operation corresponding to a drift measurement, a mechanical feature matrix for that grasping operation is synchronously collected and constructed, and aligned temporally with the drift measurement to ensure that the reflected force pattern corresponds to the specific pose error state at that time. After multiple synchronous operations, a sample mechanical feature matrix set is accumulated, and the scale and direction of the robotic arm's drift deviation under different sample mechanical feature matrices are collected and labeled to obtain a sample drift error set. For each matrix in the sample mechanical feature matrix set, the corresponding true error vector measured during grasping, containing the scale and direction of the error, is found from the robotic arm drift error monitoring data. The drift error is used as a label for the mechanical feature matrix. All these labels are collected to form the sample drift error set.

[0023] Secondly, based on machine learning, a drift error predictor is constructed. This predictor includes multiple sets of initial weights and biases. Weights are the strength coefficients of the connections between different neurons in the neural network; each weight is a learnable scalar value that determines the contribution of the output value of a neuron in a previous layer to the input value of a neuron in a subsequent layer. Bias are learnable constant terms appended to each neuron in each layer of the neural network, added to the weighted sum before passing through the activation function. Bias allows neurons to have non-zero outputs even when all inputs are zero, increasing the model's expressive power.

[0024] Specifically, the first step is to select an appropriate machine learning algorithm and construct a drift error predictor. Taking a deep neural network (DNN) as an example, a drift error predictor is constructed. DNN is a complex model based on a multilayer perceptron (MLP), which stacks multiple hidden layers between the input and output layers. Each layer performs a nonlinear transformation, progressively extracting features and abstracting representations from the input data.

[0025] The structure of a deep neural network includes an input layer, hidden layers, and an output layer. The input layer receives the input data, and the number of neurons must match the dimension of the feature matrix. The hidden layers are the core of the fully connected neural network, performing complex nonlinear transformations on the data to extract deep features. Each neuron in a hidden layer receives input from the neurons in the previous layer, and through weighted summation, bias term correction, and activation functions (using ReLU activation function), three fully connected hidden layers are designed. The output layer outputs the predicted drift error value, and the number of neurons must match the dimension of the sample drift error. If the predicted 3D translation error is used, the output layer has 3 neurons. Finally, parameter initialization is performed. Initial random values ​​are assigned to all connection weights and the bias of each neuron according to preset rules. The weights and biases are assigned very small random numbers; at this point, the model's weights and biases are randomized, making it a drift error predictor without learning capabilities.

[0026] Finally, the drift error predictor is trained and optimized using the set of sample mechanical feature matrices as training input and the set of sample drift errors as training supervision labels until the test converges, thus completing the training.

[0027] Specifically, the predicted drift error is calculated through forward propagation. Then, the loss is calculated by comparing the predicted value with the corresponding training and supervision labels, and using the loss function to calculate the loss value. Next, through backpropagation, the gradient of the loss function with respect to each weight and bias parameter of the model is calculated. This gradient reduces the loss, providing the direction and approximate magnitude of parameter adjustment. Finally, the parameters are updated using an optimizer based on the calculated error gradient, updating all model parameters. The validation loss is calculated by traversing the entire training set. The training process stops when the test converges. Finally, the parameter state of the model at this point is saved, completing the training and obtaining the drift error predictor.

[0028] For example, a deep neural network is used to train the drift error predictor model. The sample mechanical feature matrix set is used as the training input, and the sample drift error set is used as the training supervision label. The training set, validation set, and test set are divided in a 7:2:1 ratio. Training parameters are set with a learning rate of 0.001, and the Adam optimizer is used for training. Through forward propagation, the input data is weighted and summed using weights and biases, and then nonlinearly transformed using an activation function. Through multiple nonlinear transformations, the complex relationship between the input data and the target output is learned. Subsequently, the gradient information of the output error is calculated through backpropagation, and the weights and biases in the network are updated using gradient descent. Through optimization, the prediction error of the model is continuously reduced. After training, the performance of the model on the reserved validation set is evaluated. If the error loss between the predicted real-time coefficient and the expert-labeled value is reduced to within 0.5, the requirement is met, and the drift error predictor is obtained.

[0029] In this embodiment, the dynamic torque information during the grasping process is transformed into an input format suitable for machine learning model processing through a mechanical feature matrix. Furthermore, the steps of constructing a supervised learning sample set based on historical high-precision monitoring data, building and initializing the machine learning model, and using the sample data for supervised training until convergence, training and building a drift error predictor, enhance the system's redundancy and fault tolerance, improve its adaptability to different robotic arm models, loads, and working conditions, and address the long-term drift problem caused by mechanical wear, thermal deformation, etc., in traditional fixed calibration schemes.

[0030] S20: By using a binocular vision sensor configured on the robotic arm, binocular images are acquired, registration analysis is performed, and registration drift error and registration drift reliability are obtained; In this embodiment, the binocular vision sensor is a vision system assembled from two rigorously calibrated cameras at a fixed baseline distance. By simulating the principle of human stereoscopic vision, it can acquire depth information of the environment. The binocular image is a pair of images synchronously acquired by the binocular vision sensor. Registration analysis is the process of finding the corresponding matching point in the right image for each pixel in the left image in stereoscopic vision. Registration drift error is the deviation between the actual position of a target fixed in the world coordinate system and its preset, theoretical position, calculated by observing the target through the binocular vision system. The registration drift confidence level is an assessment of the reliability or confidence level of the registration drift error calculation results obtained through binocular vision.

[0031] Specifically, after the grasping operation is completed and the robotic arm moves to a reset state that does not affect observation, the binocular vision sensor is triggered to acquire the current binocular image. Subsequently, registration analysis is performed on the image, calculating the lateral difference in pixel position of the same physical point in the left and right images, while simultaneously evaluating the registration quality and generating a registration drift confidence level. Next, a pre-defined reference target is identified in the image, and its pixel coordinates are obtained. Combining the calculated disparity with the inherent intrinsic and extrinsic parameters of the binocular camera, the registration drift error is calculated.

[0032] Step S20 in the method provided in this application embodiment includes: After the robotic arm returns to the reset state after the grasping is completed, a binocular image is acquired by a binocular vision sensor configured on the robotic arm, wherein the binocular image includes a left image and a right image; The left and right images are registered to obtain the disparity and the registration drift confidence level. The two-dimensional reference coordinates of the reference target in the left image are identified. Combined with the parallax and the configuration parameters of the binocular sensor, the actual reference coordinates are calculated. The deviation distance from the preset reference coordinates is calculated to obtain the registration drift error.

[0033] In this embodiment, visual inspection is first performed after the grasping is completed to ensure that the visual measurement does not interfere with the grasping action, which has extremely high real-time requirements. In the reset state of the robotic arm, the binocular vision sensor configured on the robotic arm acquires binocular images, including the left and right images, which are synchronized in time and serve as the data basis for all subsequent visual analyses.

[0034] Secondly, the left and right images obtained above are registered, the lateral difference of the pixel position of the same physical point in the left and right images is calculated to obtain the disparity, and the credibility analysis of the registration analysis results is performed to obtain the registration drift credibility.

[0035] Finally, the two-dimensional reference coordinates of the reference target in the left image are identified. Combined with the parallax and the configuration parameters of the binocular sensor, the actual reference coordinates are calculated. The deviation distance from the preset reference coordinates is calculated to obtain the registration drift error.

[0036] In step S20 of the method provided in this application embodiment, the left and right images are registered to obtain disparity and registration drift confidence, including: Randomly select a first left pixel within the left image and extract the set of left neighboring pixels of the first left pixel; Within the right image, a first right pixel is randomly selected iteratively, and a set of right neighboring pixels is extracted. The similarity with the set of left neighboring pixels is calculated, and the right pixel with the highest similarity is selected as the first right pixel. The distance with the first left pixel is calculated to obtain the disparity of the first pixel. Continue to calculate the disparity of multiple sets of pixels, and calculate the average value to obtain the disparity; Calculate the discrete parameters of disparity for multiple sets of pixels and obtain the registration drift confidence.

[0037] In this embodiment of the application, a first left pixel is randomly selected in the left image, and the set of left neighboring pixels of the first left pixel is extracted.

[0038] Specifically, within a predefined valid area of ​​the left image, a pixel is randomly selected as the first left pixel using a random selection algorithm, and the set of its left neighboring pixels is extracted. First, the neighborhood information of the first left pixel is extracted. Assuming a predefined neighborhood window size of n×n pixels, and the first left pixel is determined as the pixel center, its top, bottom, left, and right neighbors are obtained. 2 The grayscale values ​​of n pixels, 2 The gray values ​​are organized in an orderly manner to form an n×n matrix, which serves as the set of left neighboring pixels of the first left pixel.

[0039] For example, suppose the left image is a grayscale image with a resolution of 640×480. A random selection is made within the image coordinate range (50, 50) to (590, 430), and the coordinates (100, 150) are obtained using a random number generator, which is taken as the first left pixel. Assuming the neighborhood window size is 3×3, the grayscale values ​​of 9 pixels, from row 309 to row 311 and column 309 to column 311, are read with the first left pixel (100, 150) as the center. For example, the grayscale value of the center point (100, 150) is 124, and the grayscale values ​​of its adjacent pixels are all different. These 9 pixel values ​​constitute the set of left neighboring pixels.

[0040] Next, the first right pixel is randomly selected iteratively within the right image, and the set of right neighboring pixels is extracted. The similarity with the set of left neighboring pixels is calculated, and the right pixel with the highest similarity is selected as the first right pixel. The distance with the first left pixel is calculated to obtain the disparity of the first pixel.

[0041] Specifically, pixels are randomly selected multiple times within the right image to form the first right pixel. Following the same method used to construct the left neighbor pixel set, a set of pixel values ​​extracted from a neighborhood window of the same size as the left image is constructed, forming the right neighbor pixel set. Subsequently, the similarity between the right and left neighbor pixel sets is calculated. For example, the Normalized Cross-Correlation (NCC) method can be used to calculate the similarity, where NCC describes the correlation between two vectors, windows, or samples of the same dimension. Its value ranges from -1 to 1, where -1 represents that the two vectors are uncorrelated, and 1 represents that the two vectors are correlated. The formula for calculating the similarity between the right and left neighbor pixel sets is as follows:

[0042] Where m×n is the size of the neighborhood window; The value of the left neighboring pixel. The average value of the left neighboring pixels; The value of the right neighboring pixel is randomly selected; The mean of the right neighboring pixel values ​​selected in the kth random selection; , () represents the coordinates of the corresponding pixel. The value range is [-1, 1], and the closer the value is to 1, the higher the similarity.

[0043] After calculating the similarity between the right and left neighbor pixel sets, the pixel with the highest similarity value is selected as the right pixel, which is then designated as the first right pixel. The distance between the right and left pixels is calculated, and this distance is taken as the disparity of the first pixel. For example, for a successfully matched left pixel PL(xL,yR) and right pixel PR(xR,yR), the horizontal disparity d = xL - xR is calculated. Since the stereo vision system has been calibrated, corresponding points in the left and right images have the same ordinate, so only the horizontal disparity needs to be calculated.

[0044] For example, if the coordinates of the first left pixel are PL(100,150), a 3×3 neighborhood is constructed, and the similarity with the set of right neighboring pixels is calculated using the formula. The calculated values ​​are S(1)=1.0, S(2)=0.671, S(3)=0.949, S(4)=0.812, S(5)=0.735, and the maximum value S(1)=1, corresponding to candidate point 1. Then the coordinates of the first right pixel are PR(95,150), and the disparity of the first pixel is di=100-95=5 pixels.

[0045] Next, the disparity of multiple sets of pixels is calculated, and the average value is used to obtain the disparity.

[0046] Specifically, the above process is repeated N times, with pixels randomly selected at different positions in the left image each time. For each selected left pixel, the best matching point is found in the right image through random search, and the disparity value is calculated for each instance, resulting in a disparity set: {d1, d2, ..., dN}. The discreteness parameters of the disparities of multiple sets of pixels are calculated, and the registration drift confidence level is obtained.

[0047] Subsequently, the variance of the disparity set is calculated, and the sum of 1 and the reciprocal of the variances of the disparity sets is used as the registration drift confidence level. The registration drift confidence level reflects the reliability of this binocular registration result.

[0048] For example, suppose the following matching results are obtained through 5 random samplings: left pixel coordinates, right pixel coordinates, disparities (100, 150), (95, 150), d=5; (150, 200), (143, 200), d=7; (80, 120), (73, 120), d=7; (220, 180), (213, 180), d=7; (180, 220), (175, 220), d=5. The disparity set d: {5, 7, 7, 7, 5}. The mean of the disparity set is 6.2, and the variance is 0.96, resulting in a registration drift confidence level of 1 / (1+0.96)≈0.51.

[0049] In step S20 of the method provided in this application embodiment, the two-dimensional reference coordinates of the reference target in the left image are identified. Combined with the parallax and binocular sensor configuration parameters described in the example, the actual reference coordinates are calculated. The deviation distance from the preset reference coordinates is calculated to obtain the registration drift error, including: The left image is input into the benchmark target recognizer, which identifies and outputs the two-dimensional benchmark coordinates of the benchmark target. The benchmark target recognizer is constructed based on a convolutional neural network and is trained using a sample image set and a sample two-dimensional benchmark coordinate set. The principal point coordinates, baseline distance, and focal length of the binocular sensor are obtained. Combined with the two-dimensional reference coordinates and disparity, the actual three-dimensional coordinates of the reference target are calculated and used as the actual reference coordinates. Obtain the preset three-dimensional coordinates of the reference target as the preset reference coordinates, calculate the deviation distance between the preset reference coordinates and the actual reference coordinates, and obtain the registration drift error.

[0050] In this embodiment, the left image is first input into a benchmark target recognizer, which identifies and outputs the two-dimensional benchmark coordinates of the benchmark target. The benchmark target recognizer is constructed based on a convolutional neural network and trained using a set of sample images and a set of sample two-dimensional benchmark coordinates. The benchmark target recognizer is an intelligent model specifically designed to automatically detect, locate, and output specific benchmark targets in an image. The two-dimensional benchmark coordinates are the coordinate representation of the benchmark target in the two-dimensional pixel coordinate system of the left image. They typically refer to the pixel coordinates of representative geometric feature points of the target, such as the center of a circular sign, the center point or corner point of a square sign, or feature points of a specific pattern.

[0051] Specifically, after acquiring the left image through a binocular sensor and performing preprocessing such as denoising and correction, the system uses this image as input to a pre-trained benchmark target recognizer. The target recognizer is built based on a convolutional neural network.

[0052] When a new left image is input, the neural network automatically performs forward propagation calculations, extracting features through multiple layers of convolution and pooling operations, and finally directly regressing the pixel coordinates of the target in the output layer. Compared to traditional methods, the output two-dimensional reference coordinates of the target, obtained through recognition, are more adaptable to environmental changes, have better anti-interference capabilities, and offer higher positioning accuracy.

[0053] Secondly, the principal point coordinates, baseline distance, and focal length of the binocular sensor are obtained. Combined with the two-dimensional reference coordinates and disparity, the actual three-dimensional coordinates of the reference target are calculated as the actual reference coordinates. Among them, the principal point coordinates of the binocular sensor are one of the intrinsic parameters of the camera, referring to the intersection of the camera optical axis and the image plane, and the coordinates (x, y, z) in the pixel coordinate system. The baseline distance is one of the extrinsic parameters of the binocular vision system, referring to the horizontal distance between the optical centers of the left and right cameras. The longer the baseline, the greater the disparity produced at the same distance, and the higher the ranging accuracy in theory, but the overlapping area of ​​the field of view will become smaller.

[0054] The system acquires the intrinsic geometric parameters of the binocular sensors, obtaining fixed principal point coordinates (x, y, z), focal length f, and baseline distance B. Simultaneously, the system combines these parameters with two-dimensional reference coordinates (c...). x ,c y The disparity d is calculated using the triangulation formula for binocular vision. First, the obtained three-dimensional coordinates are normalized; x equals u minus c. x Dividing by f, y equals v minus c y Divide by f, and finally calculate the z-coordinate and x, y coordinates respectively. Calculate the registration drift error based on the obtained baseline coordinates. The z-coordinate is z = (f × B) / d. The larger the z-coordinate, the farther the balloon target is, and the smaller the parallax d. The larger the baseline B or focal length f, the more sensitive the measurement. Then, calculate the x-coordinate and y-coordinate positions: x = (uc) x)×z / f,y=(vc y )×z / f, where uv is the calibration coefficient.

[0055] Finally, the preset three-dimensional coordinates of the reference target are obtained as the preset reference coordinates. The deviation distance between the preset reference coordinates and the actual reference coordinates is calculated to obtain the registration drift error.

[0056] The preset 3D coordinates of the reference target are used as the comparison benchmark in this step. The offset of the pixel coordinates relative to the principal point is used as the component deviation (ΔX, ΔY, ΔZ) in each coordinate axis direction by the ratio of depth Z to focal length f. The deviation distance between the preset reference coordinates and the actual reference coordinates is obtained by calculating the Euclidean distance.

[0057] For example, if the difference between (263.7,193,2056.8) and the preset reference coordinates (265,195,2060) is: ΔX=263.7-265=-1.3mm, ΔY=193-195=-2.0mm, ΔZ=2056.8-2060=-3.2mm, the registration drift error is calculated to be approximately 4mm.

[0058] In this embodiment, images are acquired in a reset state to ensure the uniformity and comparability of the measurement benchmark, while eliminating the interference of robot posture changes on the measurement results. Through registration analysis and benchmark target recognition, robustness to challenges such as changes in ambient lighting is achieved, enabling accurate and reliable acquisition of the spatial position of the benchmark target. The reliability of registration drift is quantified by calculating the discrete parameters of disparity, improving the accuracy of benchmark target localization. The visual measurement output error value, along with the self-evaluation of the reliability of the error value, provides a crucial quality weighting basis for subsequent multi-source information fusion.

[0059] S30: Verify the prediction drift error and registration drift error to obtain the drift consistency parameter. Combine the registration drift confidence level with the calibration drift error and label the mechanical feature matrix, prediction drift error and registration drift error to obtain the drift prediction training data set. In this embodiment, the drift consistency parameter represents the degree of consistency between the prediction drift error and the registration drift error in terms of direction and magnitude, and can be calculated using the cosine of the vector angle, the normalized dot product, or the inverse distance of the error vector; the drift prediction training data set consists of the mechanical feature matrix as input features, the prediction drift error currently output by the model, the registration drift error as a reference, and one or more annotation information reflecting the overall quality of the data set.

[0060] Specifically, the drift consistency parameter is first calculated, for example, the consistency between the prediction error vector and the visual measurement error vector. Then, the fusion confidence is calculated by combining the drift consistency parameter and the registration drift confidence. If the visual confidence is high, but the consistency with the force prediction is extremely poor, the fusion confidence of the data is also reduced. Finally, the mechanical feature matrix, prediction drift error, registration drift error, and the calculated fusion confidence are integrated to form a complete drift prediction training data set.

[0061] Step S30 in the method provided in this application embodiment includes: Calculate the similarity between the predicted drift error and the registration drift error to obtain the drift consistency parameter; The fusion confidence level is calculated based on the drift consistency parameter and the registration drift confidence level. Using the fusion confidence level, the mechanical feature matrix, prediction drift error, and registration drift error are labeled to obtain the drift prediction training data set.

[0062] Specifically, first, the predicted drift error is obtained, and then the similarity between the predicted drift error and the registration drift error is calculated to obtain the drift consistency parameter. The predicted drift error is the estimated value of the robot arm pose deviation inferred by the drift error predictor model based on the mechanical feature matrix of the grasping process.

[0063] The prediction drift error from the force model and the registration drift error from the visual measurement are obtained. Then, the ratio of the dot product of the two vectors to the product of their magnitudes is used as the cosine similarity between the two vectors, with the result ranging from [-1, 1], where 1 indicates that the directions are exactly the same. If the similarity is close to 1, it indicates that the error direction of the force prediction is highly consistent with the error direction observed by the vision; conversely, if the consistency parameter is very low, it indicates that there is a serious discrepancy between the two sensors' judgments of the current state, and one or both of them may have a problem.

[0064] For example, assuming that after one crawling cycle, the prediction drift error is (0.0, +2.5, 0.0) mm, the registration drift error is (-0.1, +2.3, -0.2) mm, the dot product of the errors is 0.0×(-0.1)+2.5×2.3+0.0×(-0.2)=5.75, the modulus of the prediction drift error is 2.5, the modulus of the registration drift error is ≈2.317, and the cosine similarity is 5.75 / (2.5×2.317)≈0.993, then the drift consistency parameter is 0.99.

[0065] Secondly, based on the drift consistency parameter and the registration drift confidence, the fusion confidence is calculated. The fusion confidence is the evaluation value of the consistency of multi-source information and the evaluation value of the quality of single-source information itself.

[0066] The product of the drift consistency parameter and the registration drift confidence is used as the fusion confidence score: Fusion Confidence Score = Drift Consistency Parameter × Registration Drift Confidence Score. If the fusion confidence score completely contradicts the force prediction, the consistency parameter will be close to 0, resulting in a very low fusion confidence score. Conversely, if the two are highly consistent, the visual measurement confidence score will be low, thus lowering the fusion confidence score.

[0067] For example, with a registration drift confidence of 0.51 and a drift consistency parameter of 0.99, the fusion confidence is approximately 0.51 × 0.99 ≈ 0.50. The resulting fusion confidence is low, indicating that the data may be unreliable.

[0068] Finally, the fused confidence score was used to label the mechanical feature matrix, prediction drift error, and registration drift error, resulting in a drift prediction training dataset. The mechanical feature matrix, prediction drift error, and registration drift error were then used as input to the model, preprocessed, and formatted before being integrated into the drift prediction training dataset. The fused confidence score was used as the quality label.

[0069] In this embodiment, a drift consistency parameter is obtained by predicting the similarity between drift error and registration drift error, effectively identifying situations where a single sensor experiences sudden failure or abnormal interference. Furthermore, by combining the drift consistency parameter and registration drift confidence, a fusion confidence score is calculated, reflecting the internal and external consistency of the data set. This confidence score is then used to label the entire data set, generating standardized data packages with clear quality labels. This provides an immediate and quantifiable basis of trust for downstream fusion decisions, enabling the system to dynamically adjust control strategies based on data quality. It establishes high-quality data selection criteria for subsequent incremental model learning, ensuring the learning quality of the system's machine learning.

[0070] S40: Analyze the consistency of the identification information between the drift prediction training data set and the cumulative drift prediction training data set to obtain the identification stability parameter. Fuse the prediction drift error and the registration drift error to obtain the processed drift prediction training data set, and perform robot arm correction control and drift error predictor update training.

[0071] In this embodiment, the annotation information of the current fusion confidence level is first analyzed to determine its consistency with the accumulated annotation information of the past, thus obtaining the label stability parameter. If the error increases normally, the similarity is high, the currently calculated registration drift error is more reliable, and its weight is greater. If it is an anomaly detection, the similarity is low, the predicted drift error is more reliable, and its weight is greater. Subsequently, the two error sources are fused based on the label stability parameter. Due to the anomaly in the current data, it may be decided to trust historical trends more, for example, assigning lower weights to visual errors. The final fusion drift deviation is calculated by weighting, forming the processed drift prediction training data set. The predicted drift error and the registration drift error are fused according to their weights, and then the current correction is performed, followed by subsequent updates to the drift error predictor.

[0072] Step S40 in the method provided in this application embodiment includes: Obtain the cumulative drift prediction training data set accumulated over a historical period; The similarity between the fusion confidence score within the drift prediction training data set and the cumulative fusion confidence score set within the cumulative drift prediction training data set is calculated to obtain the label stability parameter.

[0073] In this embodiment of the application, the cumulative drift prediction training data set accumulated over a historical period is first obtained from the historical data stored in the database or cache. This set is generated cumulatively over a historical period under various operating conditions.

[0074] Secondly, the similarity between the fusion confidence score within the drift prediction training data set and the cumulative fusion confidence score set within the cumulative drift prediction training data set is calculated to obtain the label stability parameter. Here, the cumulative fusion confidence score is the cumulative value of the fusion confidence scores in the cumulative drift prediction training data set. The similarity is calculated as 1 - |fusion confidence score - cumulative fusion confidence score| / cumulative fusion confidence score. A larger absolute value indicates a greater deviation from historical norms; a smaller absolute value indicates a greater similarity and a closer alignment with historical norms. The calculated similarity score is ultimately used as the label stability parameter.

[0075] For example, assuming the cumulative fusion confidence is 0.55, the identifier stability parameter = 1 - |0.5 - 0.55| / 0.55 ≈ 0.91, which means that the current fusion confidence is consistent with the quality level of the cumulative fusion confidence and has high credibility.

[0076] Step S40 in the method provided in this application embodiment further includes: Based on the identification stability parameters, configure the registration weights and calculate the prediction weights; Based on the prediction weights and registration weights, the prediction drift error and registration drift error are weighted and fused to obtain the fused drift bias. Combined with the fused confidence, the processed drift prediction training data set is obtained. After the cumulative drift prediction training data sets reach a preset number, the drift error predictor is trained and updated. The robotic arm is subjected to grasping correction control based on the fusion drift deviation.

[0077] In this embodiment, firstly, if the current data is generally unstable, it indicates sudden environmental interference, and sensors are typically more sensitive to environmental interference such as illumination and occlusion. Therefore, when stability is poor, the visual measurement registration should be reduced, and a lower registration weight should be configured. The label stability parameter is used as the registration weight, and the sum of the registration weight and the prediction weight is 1. Then, the prediction weight is obtained by subtracting the registration weight from 1.

[0078] Secondly, based on the prediction weight and registration weight, the prediction drift error and registration drift error are weighted and fused to obtain the fused drift bias. Combined with the fused confidence, the processed drift prediction training data set is obtained.

[0079] The weighted calculation yields the fusion drift bias = prediction weight × prediction drift error + registration weight × registration drift error. Combined with the fusion confidence, the processed drift prediction training data set is obtained.

[0080] Next, after the cumulative drift prediction training data sets reach a preset number, the drift error predictor is trained and updated.

[0081] When the cumulative number of high-quality data sets reaches a preset threshold, a model training update task is triggered. At this point, the fused confidence score is treated as new labeled data, the feature matrix as input features, and the observation error as a label for supervised learning, updating the drift error predictor. Fine-tuning is typically employed, training on new data with a small learning rate, allowing the model to learn new data patterns without forgetting existing knowledge. Through regular training and updates, the drift error predictor can continuously adapt to long-term performance drift caused by wear, temperature changes, and other factors in the robotic arm.

[0082] For example, assuming the system has a preset quantity of 50, the 50th full data set is generated, and the cumulative number reaches the preset quantity of 50. The system then loads 50 mechanical feature matrices from the new data set as input and 50 corresponding observation error vectors as labels. The model is fine-tuned with 1 / 10 of the initial learning rate and trained for 10 epochs. This training update completes, enabling the new model to more accurately predict drift errors in the current state of the robotic arm.

[0083] Finally, based on the fusion drift deviation, the robotic arm is subjected to grasping correction control. This grasping correction control compensates for the identified systematic pose deviations when the robotic arm performs subsequent grasping tasks, thereby improving grasping accuracy.

[0084] Specifically, based on the fusion drift deviation, the system needs to perform grasping correction control on the robotic arm: when the robotic arm plans the next grasping task and calculates the position of the target grasping point in space, it directly performs compensation: new spatial position = spatial position - fusion drift deviation.

[0085] In this embodiment, the fusion confidence level of the current data set is compared with the historical confidence level distribution to obtain a stability parameter, enabling the system to identify potential global and systemic anomalies. Simultaneously, the stability parameter is used to dynamically configure the fusion weights, achieving adaptive weighted fusion. A mechanism that periodically trains and updates the prediction model using accumulated high-quality data continuously adapts to the performance degradation of the robotic arm and environmental changes. Finally, feedforward-type grasping correction control is executed according to the fusion deviation to compensate for the robot's movements. This ensures robustness and accuracy of real-time control in complex and changing industrial environments, possesses adaptability to long-term drift and unknown working conditions, and achieves adaptive intelligent grasping control.

[0086] The embodiments of this application, through the above specific implementation methods, achieve the following technical effects: In this embodiment, the dynamic torque information during the grasping process is first transformed into an input format suitable for machine learning model processing using a mechanical feature matrix. Further, a supervised learning sample set is constructed based on historical high-precision monitoring data, a machine learning model is built and initialized, and supervised training is performed using the sample data until convergence. This process trains and builds a drift error predictor, enhancing the system's redundancy and fault tolerance, improving its adaptability to different robotic arm models, loads, and working conditions, and addressing the long-term drift problem caused by mechanical wear, thermal deformation, etc., in traditional fixed calibration schemes.

[0087] Secondly, by acquiring images in a reset state, the uniformity and comparability of the measurement benchmark are ensured, while eliminating the interference of robot posture changes on the measurement results. Through registration analysis and benchmark target recognition, robustness to challenges such as changes in ambient lighting is achieved, enabling accurate and reliable acquisition of the spatial position of the benchmark target. The reliability of registration drift is quantified by calculating the discrete parameters of disparity, improving the accuracy of benchmark target localization. The output error value of the visual measurement, along with the self-assessment of the reliability of the error value, provides a crucial quality weighting basis for subsequent multi-source information fusion.

[0088] Furthermore, by predicting the similarity between drift error and registration drift error, a drift consistency parameter is obtained, effectively identifying situations where a single sensor experiences sudden failure or is subject to abnormal interference. Then, combining the drift consistency parameter and registration drift confidence, a fusion confidence score is calculated, reflecting the internal and external consistency of the data set. This confidence score is used to label the entire data set, generating standardized data packages with clear quality labels. This provides a quantitative basis for downstream fusion decisions, enabling the system to dynamically adjust control strategies based on data quality. It establishes high-quality data selection criteria for subsequent incremental model learning, ensuring the learning quality of the system's machine learning.

[0089] Finally, by comparing the fusion confidence score of the current data set with the historical confidence score distribution, a stability parameter is obtained, enabling the system to identify potential anomalies. Simultaneously, using this stability parameter, dynamic weighting is configured to achieve adaptive weighted fusion. A mechanism that periodically trains and updates the prediction model using accumulated high-quality data continuously adapts to the performance degradation of the robotic arm and environmental changes. Finally, feedforward-based grasping correction control is executed according to the fusion deviation to compensate for the robot's movements. This ensures robustness and accuracy of real-time control in complex and changing industrial environments, possesses adaptability to long-term drift and unknown working conditions, and achieves adaptive intelligent grasping control.

[0090] Example 2, as Figure 2 As shown, based on the same inventive concept as the robotic arm grasping intelligent control method based on online pose correction provided in Embodiment 1, this embodiment of the invention also provides a robotic arm grasping intelligent control system based on online pose correction, including: The drift error prediction module 11 is used to detect the mechanical feature matrix during the grasping process by using a mechanical sensor configured at the end of the robotic arm, and to predict the drift error by using a drift error predictor to obtain the predicted drift error. The registration analysis module 12 is used to acquire binocular images through a binocular vision sensor configured on the robotic arm, perform registration analysis, and obtain the registration drift error and registration drift confidence. The registration confidence annotation module 13 is used to verify the prediction drift error and the registration drift error, obtain the drift consistency parameter, and, in combination with the registration drift confidence, annotate the mechanical feature matrix, the prediction drift error and the registration drift error to obtain the drift prediction training data set. The correction update module 14 is used to analyze the consistency of the identification information between the drift prediction training data set and the cumulative drift prediction training data set, obtain the identification stability parameter, fuse the prediction drift error and the registration drift error to obtain the processed drift prediction training data set, and perform update training for the robotic arm correction control and the drift error predictor.

[0091] In one embodiment, the drift error prediction module 11 is used for: By using a mechanical sensor configured at the end of the robotic arm, a mechanical feature matrix is ​​obtained during the grasping process. The mechanical feature matrix includes mechanical feature values ​​in multiple directions. The mechanical feature matrix is ​​input into the drift error predictor, and the predicted drift error is output.

[0092] The training steps for the drift error predictor include: Based on the monitoring data of robot arm drift error over a historical period, a set of sample mechanical feature matrices was collected, and the scale and direction of the drift deviation sent by the robot arm under different sample mechanical feature matrices were collected and labeled to obtain a set of sample drift errors. A drift error predictor is constructed based on machine learning, wherein the drift error predictor includes multiple sets of initial weights and biases; The drift error predictor is trained and optimized using the set of sample mechanical feature matrices as training input and the set of sample drift errors as training supervision labels until the test converges, thus completing the training.

[0093] In one embodiment, the registration analysis module 12 is used for: After the robotic arm returns to the reset state after the grasping is completed, a binocular image is acquired by a binocular vision sensor configured on the robotic arm, wherein the binocular image includes a left image and a right image; The left and right images are registered to obtain the disparity and the registration drift confidence level. The two-dimensional reference coordinates of the reference target in the left image are identified. Combined with the parallax and the configuration parameters of the binocular sensor, the actual reference coordinates are calculated. The deviation distance from the preset reference coordinates is calculated to obtain the registration drift error.

[0094] The process of registering the left and right images to obtain disparity and registration drift confidence includes: Randomly select a first left pixel within the left image and extract the set of left neighboring pixels of the first left pixel; Within the right image, a right pixel is randomly selected iteratively, and a set of right neighboring pixels is extracted. The similarity with the set of left neighboring pixels is calculated, and the right pixel with the highest similarity is selected as the first right pixel. The distance between the right pixel and the first left pixel is calculated to obtain the disparity of the first pixel. Continue to calculate the disparity of multiple sets of pixels, and calculate the average value to obtain the disparity; Calculate the discrete parameters of disparity for multiple sets of pixels and obtain the registration drift confidence.

[0095] Specifically, the two-dimensional reference coordinates of the reference target within the left image are identified. Combined with the parallax and configuration parameters of the binocular sensor, the actual reference coordinates are calculated. The deviation distance from the preset reference coordinates is calculated to obtain the registration drift error, which includes: The left image is input into the benchmark target recognizer, which identifies and outputs the two-dimensional benchmark coordinates of the benchmark target. The benchmark target recognizer is constructed based on a convolutional neural network and is trained using a sample image set and a sample two-dimensional benchmark coordinate set. The principal point coordinates, baseline distance, and focal length of the binocular sensor are obtained. Combined with the two-dimensional reference coordinates and disparity, the actual three-dimensional coordinates of the reference target are calculated and used as the actual reference coordinates. Obtain the preset three-dimensional coordinates of the reference target as the preset reference coordinates, calculate the deviation distance between the preset reference coordinates and the actual reference coordinates, and obtain the registration drift error.

[0096] In one embodiment, the registration confidence labeling module 13 is used for: The similarity between the predicted drift error and the registration drift error is calculated to obtain the drift consistency parameter; The fusion confidence level is calculated based on the drift consistency parameter and the registration drift confidence level. Using the fusion confidence level, the mechanical feature matrix, prediction drift error, and registration drift error are labeled to obtain the drift prediction training data set.

[0097] In one embodiment, the correction update module 14 is used for: Obtain the cumulative drift prediction training data set accumulated over a historical period; The similarity between the fusion confidence score within the drift prediction training data set and the cumulative fusion confidence score set within the cumulative drift prediction training data set is calculated to obtain the label stability parameter.

[0098] The process involves fusing the predicted drift error and the registration drift error to obtain a processed drift prediction training data set, which is then used for updating and training the robotic arm correction control and the drift error predictor. This includes: Based on the identification stability parameters, configure the registration weights and calculate the prediction weights; Based on the prediction weights and registration weights, the prediction drift error and registration drift error are weighted and fused to obtain the fused drift bias. Combined with the fused confidence, the processed drift prediction training data set is obtained. After the cumulative drift prediction training data sets reach a preset number, the drift error predictor is trained and updated. The robotic arm is subjected to grasping correction control based on the fusion drift deviation.

[0099] The embodiments of this application, through the above specific implementation methods, achieve the following technical effects: In this embodiment, the dynamic torque information during the grasping process is first transformed into an input format suitable for machine learning model processing using a mechanical feature matrix. Further, a supervised learning sample set is constructed based on historical high-precision monitoring data, a machine learning model is built and initialized, and supervised training is performed using the sample data until convergence. This process trains and builds a drift error predictor, enhancing the system's redundancy and fault tolerance, improving its adaptability to different robotic arm models, loads, and working conditions, and addressing the long-term drift problem caused by mechanical wear, thermal deformation, etc., in traditional fixed calibration schemes.

[0100] Secondly, by acquiring images in a reset state, the uniformity and comparability of the measurement benchmark are ensured, while eliminating the interference of robot posture changes on the measurement results. Through registration analysis and benchmark target recognition, robustness to challenges such as changes in ambient lighting is achieved, enabling accurate and reliable acquisition of the spatial position of the benchmark target. The reliability of registration drift is quantified by calculating the discrete parameters of disparity, improving the accuracy of benchmark target localization. The output error value of the visual measurement, along with the self-assessment of the reliability of the error value, provides a crucial quality weighting basis for subsequent multi-source information fusion.

[0101] Furthermore, by predicting the similarity between drift error and registration drift error, a drift consistency parameter is obtained, effectively identifying situations where a single sensor experiences sudden failure or is subject to abnormal interference. Then, combining the drift consistency parameter and registration drift confidence, a fusion confidence score is calculated, reflecting the internal and external consistency of the data set. This confidence score is used to label the entire data set, generating standardized data packages with clear quality labels. This provides a quantitative basis for downstream fusion decisions, enabling the system to dynamically adjust control strategies based on data quality. It establishes high-quality data selection criteria for subsequent incremental model learning, ensuring the learning quality of the system's machine learning.

[0102] Finally, by comparing the fusion confidence score of the current data set with the historical confidence score distribution, a stability parameter is obtained, enabling the system to identify potential anomalies. Simultaneously, using this stability parameter, dynamic weighting is configured to achieve adaptive weighted fusion. A mechanism that periodically trains and updates the prediction model using accumulated high-quality data continuously adapts to the performance degradation of the robotic arm and environmental changes. Finally, feedforward-based grasping correction control is executed according to the fusion deviation to compensate for the robot's movements. This ensures robustness and accuracy of real-time control in complex and changing industrial environments, possesses adaptability to long-term drift and unknown working conditions, and achieves adaptive intelligent grasping control.

Claims

1. A robotic arm grasping intelligent control method based on online pose correction, characterized in that, The method includes: By using a mechanical sensor configured at the end of the robotic arm, a mechanical feature matrix is ​​detected during the grasping process. A drift error predictor is then used to predict the drift error, thus obtaining the predicted drift error. By using a binocular vision sensor configured on the robotic arm, binocular images are acquired, and registration analysis is performed to obtain the registration drift error and registration drift reliability. The predicted drift error and registration drift error are verified to obtain the drift consistency parameter. Combined with the registration drift confidence, the mechanical feature matrix, the predicted drift error and the registration drift error are labeled to obtain the drift prediction training data set. The consistency of the identification information between the drift prediction training data set and the cumulative drift prediction training data set is analyzed to obtain the identification stability parameter. The prediction drift error and the registration drift error are fused to obtain the processed drift prediction training data set, which is then used for robot arm correction control and drift error predictor update training.

2. The intelligent control method for robotic arm grasping based on online pose correction according to claim 1, characterized in that, By using a mechanical sensor configured at the end of the robotic arm, a mechanical feature matrix is ​​detected during the grasping process. A drift error predictor is then used to predict the drift error, which includes: By using a mechanical sensor configured at the end of the robotic arm, a mechanical feature matrix is ​​obtained during the grasping process. The mechanical feature matrix includes mechanical feature values ​​in multiple directions. The mechanical feature matrix is ​​input into the drift error predictor, and the predicted drift error is output.

3. The intelligent control method for robotic arm grasping based on online pose correction according to claim 2, characterized in that, The training steps for the drift error predictor include: Based on the monitoring data of robot arm drift error over a historical period, a set of sample mechanical feature matrices was collected, and the scale and direction of the drift deviation sent by the robot arm under different sample mechanical feature matrices were collected and labeled to obtain a set of sample drift errors. A drift error predictor is constructed based on machine learning, wherein the drift error predictor includes multiple sets of initial weights and biases; The drift error predictor is trained and optimized using the set of sample mechanical feature matrices as training input and the set of sample drift errors as training supervision labels until the test converges, thus completing the training.

4. The intelligent control method for robotic arm grasping based on online pose correction according to claim 1, characterized in that, Binocular vision sensors mounted on a robotic arm are used to acquire binocular images, perform registration analysis, and obtain registration drift error and registration drift reliability, including: After the robotic arm returns to the reset state after the grasping is completed, a binocular image is acquired by a binocular vision sensor configured on the robotic arm, wherein the binocular image includes a left image and a right image; The left and right images are registered to obtain the disparity and the registration drift confidence level. The two-dimensional reference coordinates of the reference target in the left image are identified. Combined with the parallax and the configuration parameters of the binocular sensor, the actual reference coordinates are calculated. The deviation distance from the preset reference coordinates is calculated to obtain the registration drift error.

5. The intelligent control method for robotic arm grasping based on online pose correction according to claim 4, characterized in that, The left and right images are registered to obtain disparity, and the registration drift confidence level is obtained, including: Randomly select a first left pixel within the left image and extract the set of left neighboring pixels of the first left pixel; Within the right image, a first right pixel is randomly selected iteratively, and a set of right neighboring pixels is extracted. The similarity with the set of left neighboring pixels is calculated, and the right pixel with the highest similarity is selected as the first right pixel. The distance with the first left pixel is calculated to obtain the disparity of the first pixel. Continue to calculate the disparity of multiple sets of pixels, and calculate the average value to obtain the disparity; Calculate the discrete parameters of disparity for multiple sets of pixels and obtain the registration drift confidence.

6. The intelligent control method for robotic arm grasping based on online pose correction according to claim 4, characterized in that, Identify the two-dimensional reference coordinates of the reference target within the left image, and calculate the actual reference coordinates by combining the parallax and the configuration parameters of the binocular sensor. Calculate the deviation distance from the preset reference coordinates to obtain the registration drift error, including: The left image is input into the benchmark target recognizer, which identifies and outputs the two-dimensional benchmark coordinates of the benchmark target. The benchmark target recognizer is constructed based on a convolutional neural network and is trained using a sample image set and a sample two-dimensional benchmark coordinate set. The principal point coordinates, baseline distance, and focal length of the binocular sensor are obtained. Combined with the two-dimensional reference coordinates and disparity, the actual three-dimensional coordinates of the reference target are calculated and used as the actual reference coordinates. Obtain the preset three-dimensional coordinates of the reference target as the preset reference coordinates, calculate the deviation distance between the preset reference coordinates and the actual reference coordinates, and obtain the registration drift error.

7. The intelligent control method for robotic arm grasping based on online pose correction according to claim 1, characterized in that, The predicted drift error and registration drift error are verified to obtain drift consistency parameters. Combined with the registration drift confidence level, the mechanical feature matrix, predicted drift error, and registration drift error are labeled to obtain a drift prediction training data set, including: The similarity between the predicted drift error and the registration drift error is calculated to obtain the drift consistency parameter; The fusion confidence level is calculated based on the drift consistency parameter and the registration drift confidence level. Using the fusion confidence level, the mechanical feature matrix, prediction drift error, and registration drift error are labeled to obtain the drift prediction training data set.

8. The intelligent control method for robotic arm grasping based on online pose correction according to claim 1, characterized in that, Analyzing the consistency of the labeling information between the drift prediction training data set and the cumulative drift prediction training data set yields labeling stability parameters, including: Obtain the cumulative drift prediction training data set accumulated over a historical period; The similarity between the fusion confidence score within the drift prediction training data set and the cumulative fusion confidence score set within the cumulative drift prediction training data set is calculated to obtain the label stability parameter.

9. The intelligent control method for robotic arm grasping based on online pose correction according to claim 1, characterized in that, The predicted drift error and the registration drift error are fused to obtain a processed drift prediction training data set, which is then used for updating and training the robotic arm correction control and the drift error predictor, including: Based on the identification stability parameters, configure the registration weights and calculate the prediction weights; Based on the prediction weights and registration weights, the prediction drift error and registration drift error are weighted and fused to obtain the fused drift bias. Combined with the fused confidence, the processed drift prediction training data set is obtained. After the cumulative drift prediction training data sets reach a preset number, the drift error predictor is trained and updated. The robotic arm is subjected to grasping correction control based on the fusion drift deviation.

10. A robotic arm grasping intelligent control system based on online pose correction, characterized in that, The system is used to implement the intelligent control method for robotic arm grasping based on online pose correction as described in any one of claims 1-9, the system comprising: The drift error prediction module is used to detect the mechanical feature matrix during the grasping process by using a mechanical sensor configured at the end of the robotic arm, and then use a drift error predictor to predict the drift error. The registration analysis module is used to acquire binocular images through a binocular vision sensor configured on the robotic arm, perform registration analysis, and obtain the registration drift error and registration drift confidence. The registration confidence annotation module is used to verify the prediction drift error and the registration drift error, obtain the drift consistency parameter, and, in combination with the registration drift confidence, annotate the mechanical feature matrix, the prediction drift error and the registration drift error to obtain the drift prediction training data set. The calibration update module is used to analyze the consistency of the identification information between the drift prediction training data set and the cumulative drift prediction training data set, obtain the identification stability parameter, fuse the prediction drift error and the registration drift error to obtain the processed drift prediction training data set, and perform update training for the robotic arm calibration control and the drift error predictor.

Images