A fixture grasping system for a mechanical claw

Abstract

The present invention relates to the field of manipulator technology, and discloses a fixture grasping system for a mechanical claw, comprising: a mechanical claw, a fixture and a monitoring module, a processing module and an early warning module. The monitoring module comprises a visual acquisition unit and a clamping force acquisition unit, which respectively acquire the fixture image before grasping and the force data of the claw fingers during grasping; the early warning module receives the early warning signal and alarms. The processing module comprises an image processing unit, an acquisition unit and a calculation unit, the image processing unit calculates the diagonal coordinates of the fixture through edge detection; the acquisition unit acquires the diagonal coordinates and force data and establishes a data set and a historical data set; the calculation unit determines the threshold through the historical data set, establishes a two-dimensional matrix to analyze abnormalities, and sends an early warning signal if there are abnormal points in the matrix. The present invention realizes high-precision grasping and real-time fault detection by combining technologies such as visual monitoring, clamping force acquisition, image processing and data analysis.

Description

Technical Field

[0001] The present invention relates to the technical field of mechanical claws, and in particular to a fixture grasping system for mechanical claws. Background Art

[0002] EOAT (full name: End of Arm Tooling) is the end tool of the robot gripper, which includes the robot gripper, fixture, etc. During the operation, the robot gripper undertakes key tasks such as grasping, moving, and placing the workpiece. The grasping performance of the robot gripper plays a vital role in the efficiency of the entire production process and product quality; the fixture is a device used to position, support and fix the workpiece, which can ensure that the workpiece maintains a stable posture and position during the production process.

[0003] However, due to factors such as sensor accuracy and motor drive errors, motion control systems often fail to achieve perfect accuracy and may even damage the workpiece. In real-time control, various factors such as system delays or feedback lags may cause slight deviations when the gripper performs a predetermined action, especially in the case of long-term operation or multi-tasking. Errors may accumulate over multiple repetitive actions, ultimately leading to insufficient motion accuracy and affecting grasping accuracy.

[0004] Therefore, it is particularly important to design a high-precision grasping system for a mechanical gripper to grasp workpieces on a fixture. Summary of the invention

[0005] In view of this, the present invention proposes a fixture grasping system for a mechanical claw, aiming to solve the problem of low grasping accuracy of the mechanical claw due to various factors in the current technology.

[0006] The present invention provides a fixture grasping system for a mechanical claw, comprising a mechanical claw and a fixture, and further comprising:

[0007] Monitoring module, processing module and early warning module;

[0008] The monitoring module includes a vision acquisition unit and a clamping force acquisition unit; the vision acquisition unit is fixedly installed at the end of the mechanical claw, and the vision acquisition unit is configured to periodically acquire the image of the fixture before grasping; the clamping force acquisition unit is configured to acquire the force data of each claw of the mechanical claw when the mechanical claw grasps; the early warning module is configured to issue an alarm after receiving the early warning signal;

[0009] The processing module includes an image processing unit, a collection unit, and a calculation unit;

[0010] The image processing unit is configured to extract the edge of the jig image based on edge detection and calculate two diagonal coordinates of the jig;

[0011] The acquisition unit is configured to acquire the diagonal coordinates and force data according to a time series and establish a data set, wherein the data set includes the monitoring time and the diagonal coordinates and force data corresponding to the monitoring time; and acquire a data set when the mechanical claw grasps normally and use it as a historical data set;

[0012] The computing unit is configured to determine a coordinate threshold and a force threshold when the mechanical gripper grasps normally based on the historical data set, establish a two-dimensional data matrix based on the data set, and determine whether there is an abnormality in the mechanical gripper grasping based on the coordinate threshold and the force threshold analysis; when there is at least one abnormal point in the two-dimensional data matrix, it is determined that there is an abnormality in the mechanical gripper grasping, and an early warning signal is sent to the early warning module.

[0013] Furthermore, before the image processing unit acquires the jig image and calculates the two diagonal coordinates of the jig, it performs integrity detection on the jig image, including:

[0014] Normalizing the pixel values ​​of the jig image, and passing the normalized jig image into a convolutional neural network model, wherein the convolutional neural network model includes a plurality of convolutional layers, a pooling layer, and a fully connected layer, and the fully connected layer outputs a binary classification result;

[0015] Based on the output of the convolutional neural network model, an integrity score of the fixture image is obtained, and the integrity score is compared with a preset integrity threshold;

[0016] When the integrity score is greater than the integrity threshold, the jig image is used as an image to be analyzed;

[0017] When the integrity score is less than or equal to the preset integrity threshold, the jig image is reacquired.

[0018] Further, when the jig image is reacquired, the acquisition period of the jig image is shortened and adjusted, and the adjusted acquisition period satisfies the following relationship:

[0019] ;

[0020] in, is the adjusted acquisition period, is the acquisition cycle before adjustment, is the shortest period allowed by the system, Score integrity. is the integrity threshold.

[0021] Furthermore, the image processing unit extracts the edge of the jig image based on edge detection and calculates two diagonal coordinates of the jig, including:

[0022] Performing denoising on the acquired jig image, wherein the denoising is to use Gaussian filtering or median filtering to remove noise in the image;

[0023] Based on the Canny edge detection algorithm, the edges in the fixture image are extracted to obtain the edge information of the fixture contour;

[0024] Based on the edge information, two diagonal corners are selected and the coordinates of the two diagonal corners of the fixture image are calculated using a geometric algorithm.

[0025] Further, when two diagonal angles are selected, it includes:

[0026] Take every two diagonals as a diagonal group, calculate the accuracy of all the diagonals and determine whether the accuracy of the two diagonals in the diagonal group is greater than the preset accuracy threshold. If so, calculate the diagonal coordinates of the two diagonals in the diagonal group. If not, select the diagonal group containing the diagonal with the highest accuracy as the coordinate calculation group, and determine the accuracy of the two diagonals in the coordinate calculation group:

[0027] When only one diagonal angle in the coordinate calculation group has an accuracy greater than the accuracy threshold, the diagonal angle is selected as the reference angle, and the diagonal coordinates of another diagonal angle in the coordinate calculation group are calculated based on the reference angle; when there is no accuracy greater than the accuracy threshold in the coordinate calculation group, the image processing unit re-denoises the acquired fixture image and calculates the diagonal coordinates.

[0028] Furthermore, when calculating the accuracy, it includes:

[0029] The ideal contour and the actual contour when the mechanical claw grasps the fixture are obtained, and the Hausdorff distance between the ideal contour and the actual contour is calculated. The accuracy has the following relationship:

[0030] ;

[0031] in, For accuracy, represents the actual contour, represents the ideal contour, Represents the Hausdorff distance between the ideal contour and the actual contour.

[0032] Furthermore, when the calculation unit determines the coordinate threshold and the force threshold when the mechanical claw grasps normally according to the historical data set, it includes:

[0033] Obtain the pre-input standard diagonal coordinates and the standard force data of each claw of the mechanical claw, and the horizontal and vertical coordinates of the coordinate threshold are in the following range:

[0034] ;

[0035] ;

[0036] The stress threshold is in the following range:

[0037] ;

[0038] in, and are the means of all horizontal and vertical coordinates in the historical data set, and are the standard deviations of all horizontal and vertical coordinates in the historical data set, is the mean of all force data of the i-th mechanical gripper finger in the historical data set, is the standard deviation of all force data of the i-th mechanical gripper finger in the historical data set, k represents the tolerance range, 1≤k≤3.

[0039] Further, when judging whether there is any abnormality in the gripping of the mechanical claw according to the coordinate threshold and the force threshold analysis, it includes:

[0040] Extracting the diagonal coordinates and the force data of the claw finger corresponding to the monitoring time from the two-dimensional data matrix; judging whether the current diagonal coordinates are within the range of the coordinate threshold, if the abscissa and / or ordinate exceeds the range of the coordinate threshold, recording the monitoring time and the corresponding diagonal coordinates and marking them as abnormal points;

[0041] The force data of each claw finger is detected to determine whether the force data is within the range of the force threshold. If the force data of any claw finger exceeds the range of the force threshold, the monitoring time and the corresponding force data are recorded as abnormal points.

[0042] Furthermore, the early warning module is also configured to: count the number of abnormal points and issue graded alarms according to the number of abnormal points.

[0043] Further, the image processing unit sets a first upper limit of the number of times of reacquiring the jig image and a second upper limit of the number of times of re-performing denoising and calculating the diagonal coordinates;

[0044] When the number of times the jig image is reacquired meets the first number upper limit or when the number of times the denoising process is re-performed and the diagonal coordinates are calculated meets the preset second number upper limit, a warning signal is sent to the warning module.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] The visual acquisition unit and clamping force acquisition unit of the monitoring module can obtain the fixture image and force data in real time. Combined with image processing and data analysis technology, the grasping status of the mechanical claw can be detected in time to ensure more accurate grasping action.

[0047] By establishing historical data sets and setting coordinate and force thresholds, the system can dynamically analyze data during the grasping process, detect abnormal situations in a timely manner, and issue alarms through the early warning module to effectively prevent potential failures or misoperations.

[0048] When it is detected that the integrity of the fixture image does not meet the requirements, the system can automatically adjust the image acquisition cycle and re-process the image to ensure that the image information during the grasping process always meets the requirements, thereby improving the reliability of the grasping process.

[0049] By performing image denoising, edge detection, and diagonal coordinate calculations, and using Hausdorff distance accuracy judgment, high-precision identification of the fixture is ensured and the accuracy of grasping is optimized.

[0050] Based on the number and type of abnormal points, the system can issue graded alarms to help operators respond to problems in the grasping process more quickly and accurately, thereby improving operational safety and production efficiency.

[0051] By timely monitoring deviations and errors in the grasping process, the error accumulation caused by system delays or inaccurate control can be reduced, ensuring stable accuracy during long-term and multi-task operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present invention. Moreover, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0053] Figure 1 A functional block diagram of a fixture grasping system for a mechanical claw provided in an embodiment of the present invention;

[0054] Figure 2 A three-dimensional schematic diagram of a fixture provided in an embodiment of the present invention.

[0055] Among them, 1. Jig; 2. Material hole. DETAILED DESCRIPTION

[0056] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features described in the embodiments can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0057] like Figure 1 As shown, an embodiment of the present invention provides a fixture grasping system for a mechanical claw, including a mechanical claw and a fixture, and further comprising:

[0058] Monitoring module, processing module and early warning module;

[0059] The monitoring module includes a visual acquisition unit and a clamping force acquisition unit; the visual acquisition unit is fixedly installed at the end of the mechanical claw, and the visual acquisition unit is configured to periodically acquire the image of the fixture before grasping; the clamping force acquisition unit is configured to acquire the force data of each claw of the mechanical claw when the mechanical claw grasps; the early warning module is configured to issue an alarm after receiving the early warning signal;

[0060] The processing module includes an image processing unit, an acquisition unit, and a calculation unit;

[0061] The image processing unit is configured to extract the edge of the jig image based on edge detection and calculate two diagonal coordinates of the jig;

[0062] The acquisition unit is configured to acquire diagonal coordinates and force data according to a time series and establish a data set, wherein the data set includes the monitoring time and the diagonal coordinates and force data corresponding to the monitoring time; and acquire a data set when the mechanical claw grasps normally and use it as a historical data set;

[0063] The computing unit is configured to determine the coordinate threshold and force threshold when the mechanical gripper grasps normally based on the historical data set, establish a two-dimensional data matrix based on the data set, and determine whether there is an abnormality in the mechanical gripper grasping based on the coordinate threshold and force threshold analysis; when there is at least one abnormal point in the two-dimensional data matrix, it is determined that there is an abnormality in the mechanical gripper grasping, and an early warning signal is sent to the early warning module.

[0064] It is understandable that it should be noted that the visual acquisition unit and the clamping force acquisition unit in the monitoring module can obtain the image of the fixture and the force data of the mechanical claw when grasping in real time. The visual acquisition unit can regularly capture the image of the fixture to ensure the correct positioning before grasping, while the clamping force acquisition unit continuously tracks the force condition of the mechanical claw. This provides a direct basis for judging the accuracy and stability of grasping.

[0065] The processing module continuously collects and processes data from the grasping process based on time series through the image processing unit, acquisition unit and calculation unit. By performing edge detection and coordinate calculation on the fixture image, the system can extract key information of the fixture and store and analyze it together with the force data. These data construct a historical data set that can provide a reliable reference for subsequent grasping accuracy analysis.

[0066] The computing unit analyzes the gripping accuracy and force of the mechanical claw through historical data sets, and evaluates whether the current gripping is abnormal based on the pre-set coordinate threshold and force threshold. When an abnormal point appears in the data matrix, the system can accurately determine that there is a problem with the gripping and issue an alarm through the early warning module. This real-time early warning mechanism can detect potential abnormalities at an early stage, thereby avoiding damage or failure caused by inaccurate grasping or insufficient force.

[0067] The system's early warning module can not only issue an alarm, but also help the operator adjust the grasping strategy in time to ensure that the grasping process of the mechanical claw and fixture is more accurate and safe. Abnormal points in the grasping process are marked and alarmed in time, effectively reducing operational risks and improving the stability and efficiency of the grasping process.

[0068] The system combines vision, mechanical sensing and data analysis technology to automatically adjust the grasping strategy and handle different grasping scenarios. In actual use, the system can adaptively adjust the working cycle, threshold setting, etc. according to different situations, enhancing the overall intelligence level and automation capabilities and improving work efficiency.

[0069] like Figure 2 As shown, it is a three-dimensional schematic diagram of the jig. The jig 1 is provided with a plurality of material holes 2 for fixing and placing raw materials. The image acquisition unit is located directly above the jig 1. The jig image at this angle is a regular rectangle.

[0070] In some embodiments of the present application, before the image processing unit acquires the jig image and calculates the two diagonal coordinates of the jig, the image of the jig is subjected to integrity detection, including:

[0071] The pixel values ​​of the fixture image are normalized, and the normalized fixture image is passed to the convolutional neural network model. The convolutional neural network model includes several convolutional layers, pooling layers, and fully connected layers. The output of the fully connected layer is a binary classification result.

[0072] Based on the output of the convolutional neural network model, the integrity score of the fixture image is obtained, and the integrity score is compared with the pre-set integrity threshold;

[0073] When the integrity score is greater than the integrity threshold, the jig image is used as the image to be analyzed;

[0074] When the integrity score is less than or equal to the preset integrity threshold, the fixture image is reacquired.

[0075] It should be noted that the integrity detection of the fixture image can be specifically:

[0076] Image normalization: First, obtain the fixture image to be processed. The image may have different brightness and contrast, so it needs to be standardized. Scale each pixel value to the interval [0,1] to eliminate the influence of factors such as different image brightness and exposure, and ensure the consistency of image data input to subsequent processing stages. The normalized image will be more suitable for inputting into the neural network for feature extraction.

[0077] Use convolutional neural network (CNN) for integrity detection: Input the normalized image into a preset convolutional neural network model, which can be pre-trained or custom designed. The convolutional neural network consists of multiple convolutional layers, pooling layers, and fully connected layers. The specific processing flow is as follows:

[0078] Convolutional layer: Extract important features in the image, such as edges, shapes, textures, etc. through convolution operations.

[0079] Pooling layer: The pooling operation is used to reduce the size of the feature map, thereby reducing the amount of computation and preventing overfitting.

[0080] Fully connected layer: Comprehensively process the extracted features and output a binary classification result, indicating whether the image is intact. The output result is a probability value, usually between 0 and 1.

[0081] Integrity score and threshold comparison: After being processed by the convolutional neural network, a integrity score is obtained. For example, the score can be 0.8, indicating that the image is likely to be intact.

[0082] The score is compared with a preset integrity threshold, which is usually determined based on experience or data. If the score is higher than the threshold (such as 0.8), the fixture image is considered intact and the subsequent grasping operation can continue.

[0083] If the score is less than or equal to the threshold, it means that the image may have quality problems, such as blur or occlusion, and the image needs to be acquired again.

[0084] Re-acquire the image: When the integrity score of the image is less than or equal to the preset integrity threshold, the image needs to be re-acquired.

[0085] When reacquiring images, the acquisition cycle can be adjusted to be shorter. This means that the system will try to capture a clear image more frequently until the quality requirement is met. If a satisfactory image cannot be acquired after multiple attempts, the image quality can be improved by increasing the lighting, adjusting the camera focus, or reducing the background noise.

[0086] Termination condition: To prevent the system from falling into an endless loop due to repeated image failures, a maximum number of retries can be set (for example, 3). If the number of times the image is re-acquired exceeds the set maximum number of retries, the system will stop the image acquisition process and trigger an alarm to inform the operator that there is an image quality problem or other fault.

[0087] Alarm mechanism: When an image failure is detected and the number of retries reaches the upper limit, the system will send an alarm signal through the early warning module to prompt the operator to check or adjust the equipment.

[0088] It is understandable that before processing the fixture image, the system first performs an integrity check on the image. This is to ensure that the quality and clarity of the input image meet the analysis requirements. By normalizing the pixel values ​​of the image, it is ensured that factors such as the image's illumination and contrast will not affect the subsequent analysis, providing a more stable and consistent input. The normalized image is passed to the convolutional neural network (CNN) model for processing. CNN can automatically extract features from the image and classify them through a combination of multiple convolutional layers, pooling layers, and fully connected layers. The binary classification results output by the fully connected layer can determine whether the image meets the integrity standards, that is, whether the image is clear enough, without damage or deformation, and suitable for subsequent capture analysis. The system compares the integrity score output by the CNN with the preset integrity threshold. When the integrity score of the image is higher than the set threshold, the system considers the image quality to be qualified and continues the subsequent coordinate calculation and capture analysis; when the score is lower than the threshold, the system automatically triggers the image re-acquisition mechanism to avoid the accuracy of subsequent analysis being affected by image quality problems. This mechanism can effectively eliminate low-quality images and ensure that each processed image has sufficient clarity and accuracy. This is crucial to the accuracy and reliability of the grasping system, especially in high-precision grasping tasks, where the integrity of the image is a prerequisite for ensuring grasping accuracy. When the system encounters unqualified image quality, it can automatically re-acquire the fixture image and ensure that the final image meets the analysis requirements by adjusting the acquisition cycle or repeating the acquisition process. This adaptive adjustment mechanism can maintain the efficiency and accuracy of the system under different environments and conditions.

[0089] In some embodiments of the present application, when reacquiring the jig image, the acquisition period of the jig image is shortened and adjusted, and the adjusted acquisition period satisfies the following relationship:

[0090] ;

[0091] in, is the adjusted acquisition period, is the acquisition cycle before adjustment, is the shortest period allowed by the system, Score integrity. is the integrity threshold.

[0092] It should be noted that by dynamically adjusting the image acquisition cycle according to the image integrity score, the system can quickly respond to low-quality images and improve the efficiency of re-acquiring qualified images. Combined with the relationship between the image integrity score and the integrity threshold, the acquisition resources are reasonably allocated to ensure that the acquisition cycle matches the actual needs and avoid waste of resources. The shortest acquisition cycle limit is introduced to ensure that the system will not be overloaded due to frequent acquisition in the case of extremely poor image quality, thereby improving the reliability and stability of the system. Through intelligent adjustment of the cycle, the number of meaningless repeated acquisitions is reduced, and the speed of acquiring images that meet the requirements is accelerated, thereby improving the work efficiency of the entire system.

[0093] In the case of low-quality images, the acquisition cycle is shortened, and the system can acquire and process new images faster, reducing delays caused by unqualified images. By dynamically adjusting the cycle, the system can optimize resource allocation, avoid the increase in load caused by excessively frequent acquisition, and reduce unnecessary power consumption. Set the shortest acquisition cycle limit to avoid the impact of excessive acquisition frequency on hardware devices and system performance, and improve the robustness and fault tolerance of the system. Dynamic adjustment of the acquisition cycle helps to acquire qualified images in a short time and reduce the risk of reduced overall system efficiency due to unqualified images. The system can be adapted to different application scenarios and performance requirements by adjusting parameters (initial cycle, shortest cycle, integrity threshold), making it more flexible and adaptable. When the image quality is low, shorten the acquisition cycle to improve efficiency, while ensuring that the acquisition frequency is not lower than the safe value allowed by the system, achieving a balance between efficiency and safety.

[0094] In some embodiments of the present application, the image processing unit extracts the edge of the jig image based on edge detection and calculates two diagonal coordinates of the jig, including:

[0095] Perform denoising on the acquired fixture image, where the denoising is to use Gaussian filtering or median filtering to remove noise in the image;

[0096] Based on the Canny edge detection algorithm, the edges in the fixture image are extracted to obtain the edge information of the fixture contour;

[0097] Based on the edge information, two diagonal corners are selected and the two diagonal coordinates of the fixture image are calculated using a geometric algorithm.

[0098] It is understandable that in image processing, noise is a common interference factor, which will affect the subsequent edge detection and feature extraction. Therefore, denoising is a necessary preprocessing step. Two commonly used denoising methods include:

[0099] Gaussian filtering: Gaussian filtering is a linear smoothing filtering method that performs convolution operations on the image and uses a Gaussian function to perform weighted averaging on each pixel to remove high-frequency noise in the image. Gaussian filtering has a good effect in removing Gaussian noise (such as noise introduced during image transmission).

[0100] Median filtering: Median filtering is a nonlinear filtering method that removes impulse noise (such as salt and pepper noise) from an image by replacing the current pixel value with the median of the neighborhood around each pixel. This method is particularly effective when processing images with obvious isolated noise points. These denoising methods ensure that the image is clearer in subsequent processing and reduce the probability of false recognition during edge detection.

[0101] Edge detection is a technique used to extract object boundaries in image processing. The Canny algorithm is a classic and efficient edge detection algorithm, which is often used for image segmentation and object recognition. The steps of the Canny algorithm include:

[0102] Noise suppression: Preprocess the image through Gaussian filtering to reduce the impact of noise.

[0103] Gradient calculation: Calculate the gradient of each pixel in the image. Usually use the Sobel operator to calculate the gradient in the horizontal and vertical directions to identify edge changes.

[0104] Non-maximum suppression: Refine the gradient amplitude, retain the edges of the local maximum, and suppress other inconspicuous edges.

[0105] Dual threshold detection and edge connection: set high and low thresholds, pixels below the low threshold are suppressed, pixels above the high threshold are considered edge pixels, and edge pixels between the two thresholds are connected based on whether the adjacent edge pixels are strong edges, to further determine the final edge.

[0106] Through these steps, the Canny edge detection algorithm can effectively extract significant edges in the image and reduce the false detection rate, thereby accurately extracting the contour of the fixture.

[0107] After edge detection, the contour information in the image has been extracted. The next task is to select two diagonals from these edge information and calculate their coordinates. Usually, geometric algorithms are used for coordinate calculation. The specific process is as follows:

[0108] Select diagonal points: In the jig outline, two diagonal points related to the jig shape are often selected as positioning references. The diagonal points are usually the two largest or smallest edge points in the image, which can represent the extreme positions of the jig.

[0109] Coordinate calculation by geometric algorithm: The two diagonal coordinates are accurately determined by geometric methods, such as using edge information in the image to calculate the farthest point or by rectangle fitting. The geometric algorithm uses the relative position relationship of the edge points in the image to determine the best diagonal coordinates. The minimum enclosing rectangle algorithm or the least squares method is usually used to fit the diagonal point position.

[0110] Through the above steps, the image processing unit can effectively extract two diagonal coordinates from the fixture image, which can be used as a reference or benchmark for subsequent grasping operations. Accurate diagonal coordinates provide accurate positioning information for the gripper to ensure the accuracy of the grasping operation and avoid grasping failure or damage to the fixture.

[0111] In some embodiments of the present application, when two diagonal angles are selected, the method includes:

[0112] Take every two diagonals as a diagonal group, calculate the accuracy of all diagonals and determine whether there is a situation where the accuracy of both diagonals in the diagonal group is greater than the preset accuracy threshold. If so, calculate the diagonal coordinates of the two diagonals in the diagonal group. If not, select the diagonal group containing the diagonal with the highest accuracy as the coordinate calculation group and determine the accuracy of the two diagonals in the coordinate calculation group:

[0113] When the accuracy of only one diagonal angle in the coordinate calculation group is greater than the accuracy threshold, the diagonal angle is selected as the reference angle, and the diagonal coordinates of another diagonal angle in the coordinate calculation group are calculated based on the reference angle; when there is no accuracy greater than the accuracy threshold in the coordinate calculation group, the image processing unit re-denoises the acquired fixture image and calculates the diagonal coordinates.

[0114] It should be noted that by introducing the precision calculation and screening mechanism, it is ensured that only diagonal groups that meet the precision requirements are used for coordinate calculation, thus avoiding calculation errors caused by low-precision diagonal points.

[0115] When the accuracy of the diagonal group does not meet the requirements, the coordinate results are further optimized through the reference angle or recalculation mechanism to improve the overall accuracy. When the accuracy of all diagonal groups does not reach the set threshold, the system can automatically trigger the re-denoising and edge detection processing of the fixture image to ensure that reliable diagonal coordinate results can be output even in complex or low-quality image environments.

[0116] To address the problem of insufficient accuracy in complex scenarios, a layer-by-layer screening and compensation mechanism is used to reduce the possibility of processing failure.

[0117] In some embodiments of the present application, when calculating the accuracy, it includes:

[0118] Get the ideal contour and actual contour when the robot grips the fixture, calculate the Hausdorff distance between the ideal contour and the actual contour, and the accuracy has the following relationship:

[0119] ;

[0120] in, For accuracy, represents the actual contour, represents the ideal contour, Represents the Hausdorff distance between the ideal contour and the actual contour.

[0121] It should be noted that by calculating the Hausdorff distance between the ideal contour and the actual contour, the difference between the two is quantified in the form of accuracy.

[0122] The Hausdorff distance can effectively measure the maximum deviation between two contours, and its result directly reflects the accuracy of image processing during fixture grasping.

[0123] According to the formula, the smaller the Hausdorff distance is, that is, the closer the actual contour is to the ideal contour, the closer the accuracy is to 1, indicating that the positioning and processing results of the grasping are highly accurate.

[0124] When the Hausdorff distance is large, the system can determine whether the current processing needs further optimization (such as reacquiring the image or adjusting the denoising algorithm) based on the accuracy value. The calculation method using the Hausdorff distance can show good adaptability in various fixture scenes with complex shapes, thereby ensuring the stability and reliability of the calculation results. By dynamically evaluating the accuracy, the system can adjust the grasping action in real time to reduce the possibility of grasping failure or error accumulation. The quantification of accuracy provides data support for the subsequent selection and calculation of diagonal coordinates, making the decision more scientific. High-precision results can be used directly for grasping behavior, while low-precision results can guide the system to perform secondary optimization to improve the success rate of grasping.

[0125] Calculated by:

[0126]

[0127] in, Is the calculated contour The distance from each point p to the nearest point q, and then take the maximum value of all these minimum distances; Is the calculated contour The distance from each point q to the nearest point p, and then take the maximum of all these minimum distances.

[0128] In some embodiments of the present application, when the computing unit determines the coordinate threshold and the force threshold when the mechanical claw grasps normally according to the historical data set, it includes:

[0129] Obtain the pre-input standard diagonal coordinates and the standard force data of each claw of the mechanical claw. The horizontal and vertical coordinates of the coordinate threshold are in the following ranges:

[0130] ;

[0131] ;

[0132] The stress threshold is in the following range:

[0133] ;

[0134] in, and are the means of all horizontal and vertical coordinates in the historical data set, and are the standard deviations of all horizontal and vertical coordinates in the historical data set, is the mean of all force data of the i-th mechanical gripper finger in the historical data set, is the standard deviation of all force data of the i-th mechanical gripper finger in the historical data set, k represents the tolerance range, 1≤k≤3.

[0135] It should be noted that dynamic threshold setting is achieved based on data statistical analysis. By obtaining the horizontal coordinate, vertical coordinate and force data in the historical data set, the mean and standard deviation of each parameter are calculated to generate a dynamic threshold that is highly matched with the actual grasping process. This method based on data statistics avoids the problem of insufficient applicability that may be caused by fixed thresholds, enables the system to adapt to changes under different working conditions, and improves the adaptability and robustness of the grasping operation. The two-way interval restriction ensures that the system can tolerate random fluctuations within a certain range and quickly identify abnormal values ​​that are obviously beyond the normal range. For example, force data exceeding the tolerance interval may indicate that the grasping is too tight or too loose, thereby triggering an early warning to prevent grasping failure. The combined use of the mean and standard deviation avoids the problem of being too sensitive or inflexible due to simple threshold setting. Parameter changes within the normal fluctuation range will not trigger false alarms, and abnormal behaviors beyond the interval can be quickly captured, thereby achieving more accurate fault detection. This design can significantly reduce false alarms caused by external interference while ensuring the safety of grasping behavior. During the grasping process, the coordinated restrictions of the horizontal and vertical coordinates and the force data can effectively ensure the alignment accuracy and clamping stability of the fixture during the grasping process. For example: if the horizontal and vertical coordinates deviate from the standard range, it may indicate that the position of the fixture is abnormal, and the system can adjust the grasping operation in time to prevent the grasping deviation. If the force data is too high or too low, it may indicate the risk of damage caused by too tight clamping of the fixture, or the risk of falling off caused by too loose clamping. By making adjustments through early warning, the safety of the grasping operation is effectively improved.

[0136] The use of standardized threshold generation methods can be quickly applied to different types of fixture grasping tasks. Whether it is a smaller precision fixture or a larger heavy fixture, just input the corresponding historical data set to quickly generate a suitable threshold range to ensure the reliability of the grasping operation. This versatility makes the system easier to use and more efficient in actual industrial scenarios. The precise threshold setting of coordinates and forces can effectively avoid fixture damage or mechanical claw wear caused by excessive grasping or deviation grasping, thereby extending the service life of equipment and fixtures, reducing maintenance costs, and improving economic benefits.

[0137] This method demonstrates a high level of intelligence through the combination of data statistics and dynamic adjustment, and can adaptively set thresholds and optimize grasping operations without human intervention. This intelligent feature not only improves the work efficiency of the system, but also reduces the need for human intervention during the operation. This method significantly improves the reliability, stability and intelligence level of the mechanical gripper grasping system through dynamic threshold setting, multi-parameter collaborative restrictions and flexible tolerance range adjustment, while enhancing adaptability and scalability, providing reliable technical support for practical industrial applications.

[0138] In some embodiments of the present application, when judging whether there is an abnormality in the gripping of the mechanical claw according to the coordinate threshold and the force threshold analysis, it includes:

[0139] Extract the diagonal coordinates and force data of the claw finger corresponding to the monitoring time from the two-dimensional data matrix; determine whether the current diagonal coordinates are within the range of the coordinate threshold. If the horizontal coordinate and / or the vertical coordinate exceeds the range of the coordinate threshold, record the monitoring time and the corresponding diagonal coordinates and mark them as abnormal points;

[0140] The force data of each claw finger is detected to determine whether the force data is within the force threshold range. If the force data of any claw finger exceeds the force threshold range, the monitoring time and the corresponding force data are recorded as abnormal points.

[0141] It should be noted that, based on the historical data set, the various parameters of the robot gripper (such as diagonal coordinates, force data, etc.) are recorded in a time series manner. These data will be stored in the form of a two-dimensional data matrix. Each data point in the matrix contains the monitoring time, the corresponding diagonal coordinates, and the force data of each claw of the robot gripper. For the grasping data at each moment, the system needs to extract the diagonal coordinates and force data of each claw corresponding to that time point from the matrix to provide a basis for subsequent analysis.

[0142] According to the preset coordinate thresholds (the upper and lower limits of the horizontal and vertical coordinates), check whether the currently extracted diagonal coordinates are within these ranges. This process involves the following steps: Compare the horizontal coordinate of the current diagonal coordinate with the range of the preset coordinate threshold (e.g., whether the horizontal coordinate is between a specific minimum and maximum value). Compare the vertical coordinate of the current diagonal coordinate with the range of the preset coordinate threshold (e.g., whether the vertical coordinate is between a specific minimum and maximum value). If the value of the horizontal or vertical coordinate exceeds the preset threshold range, the grabbing operation is considered abnormal. At this time, the system will record the current monitoring time and the diagonal coordinate that exceeds the threshold, and mark it as an abnormal point.

[0143] Each gripper finger has a corresponding threshold range of force data, usually derived from statistical analysis of historical data sets. These force thresholds define the appropriate force range required for the gripper to grasp.

[0144] Check the force data of each claw finger:

[0145] If the force value of a certain finger exceeds the force threshold, it means that the gripping force of the finger is abnormal, which may lead to grasping failure or damage to the fixture. The system will record the current monitoring time and the force data that exceeds the force threshold and mark it as an abnormal point. It should be noted that if the force data of any finger exceeds the threshold, it is considered that there is an abnormality in the grasping and the corresponding alarm or response mechanism will be triggered.

[0146] By determining whether each data point (coordinate and force data) during the gripping process of the robot gripper meets the predetermined threshold range, the system can identify abnormal situations in real time. The recording and marking of abnormal points helps to promptly detect possible deviations or problems in the gripping process, and then take preventive measures. If the force of multiple claws is abnormal or the diagonal coordinates are out of bounds, it may indicate that the gripping posture, force or shape of the gripper has changed abnormally. The system will mark the monitoring point as an abnormal point and can trigger an early warning mechanism or adjust the operation strategy.

[0147] By comparing the coordinate and force data with the preset threshold, the system can accurately determine whether there is a grasping anomaly in real time, avoiding the deficiencies and misjudgments of manual inspection. Improve the grasping success rate: Anomaly detection helps to detect problems in the grasping process early, avoiding damage to the fixture or grasping failure caused by grasping anomalies, thereby improving the overall grasping accuracy and success rate of the system. Data-driven feedback mechanism: By monitoring the data in real time, the system can adjust the operating parameters in time and optimize the grasping strategy of the mechanical claw to ensure that it can work effectively in different environments or conditions.

[0148] In some embodiments of the present application, the early warning module is further configured to: count the number of abnormal points and issue graded alarms according to the number of abnormal points.

[0149] In some embodiments of the present application, the image processing unit sets a first upper limit of the number of times of reacquiring the jig image and a second upper limit of the number of times of re-performing the denoising process and calculating the diagonal coordinates;

[0150] When the number of times the jig image is re-acquired meets the first number upper limit or when the number of times the denoising process is re-performed and the diagonal coordinates are calculated meets the preset second number upper limit, an early warning signal is sent to the early warning module.

[0151] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A fixture grasping system for a mechanical claw, comprising a mechanical claw and a fixture, characterized in that: Also includes: Monitoring module, processing module and early warning module; The monitoring module includes a vision acquisition unit and a clamping force acquisition unit; the vision acquisition unit is fixedly installed at the end of the mechanical claw, and the vision acquisition unit is configured to periodically acquire the image of the fixture before grasping; the clamping force acquisition unit is configured to acquire the force data of each claw of the mechanical claw when the mechanical claw grasps; The warning module is configured to issue an alarm upon receiving a warning signal; The processing module includes an image processing unit, a collection unit, and a calculation unit; The image processing unit is configured to extract the edge of the jig image based on edge detection and calculate two diagonal coordinates of the jig; The acquisition unit is configured to acquire the diagonal coordinates and force data according to a time series and establish a data set, wherein the data set includes the monitoring time and the diagonal coordinates and force data corresponding to the monitoring time; and acquire a data set when the mechanical claw grasps normally and use it as a historical data set; The computing unit is configured to determine a coordinate threshold and a force threshold when the mechanical claw grasps normally according to the historical data set, establish a two-dimensional data matrix according to the data set, and determine whether there is an abnormality in the mechanical claw grasping according to the coordinate threshold and the force threshold analysis; When there is at least one abnormal point in the two-dimensional data matrix, it is determined that there is an abnormality in the grasping of the mechanical claw, and an early warning signal is sent to the early warning module.

2. The fixture grasping system for a mechanical claw according to claim 1, characterized in that: Before the image processing unit acquires the jig image and calculates the two diagonal coordinates of the jig, the image processing unit performs integrity detection on the jig image, including: Normalizing the pixel values ​​of the jig image, and passing the normalized jig image into a convolutional neural network model, wherein the convolutional neural network model includes a plurality of convolutional layers, a pooling layer, and a fully connected layer, and the output of the fully connected layer is a binary classification result; Based on the output of the convolutional neural network model, an integrity score of the fixture image is obtained, and the integrity score is compared with a preset integrity threshold; When the integrity score is greater than the integrity threshold, the jig image is used as an image to be analyzed; When the integrity score is less than or equal to the preset integrity threshold, the jig image is reacquired.

3. The fixture grasping system for a mechanical claw according to claim 2, characterized in that: When the jig image is reacquired, the acquisition period of the jig image is shortened and adjusted, and the adjusted acquisition period satisfies the following relationship: ； in, is the adjusted acquisition period, is the acquisition cycle before adjustment, is the shortest period allowed by the system, Score integrity. is the integrity threshold.

4. The fixture grasping system for a mechanical claw according to claim 3, characterized in that: The image processing unit extracts the edge of the jig image based on edge detection and calculates two diagonal coordinates of the jig, including: Performing denoising on the acquired jig image, wherein the denoising is to use Gaussian filtering or median filtering to remove noise in the image; Based on the Canny edge detection algorithm, the edges in the fixture image are extracted to obtain the edge information of the fixture contour; Based on the edge information, two diagonal corners are selected and the coordinates of the two diagonal corners of the fixture image are calculated using a geometric algorithm.

5. The fixture grasping system for a mechanical claw according to claim 4, characterized in that: When two opposite corners are selected, this includes: Take every two diagonals as a diagonal group, calculate the accuracy of all the diagonals and determine whether the accuracy of the two diagonals in the diagonal group is greater than the preset accuracy threshold. If so, calculate the diagonal coordinates of the two diagonals in the diagonal group. If not, select the diagonal group containing the diagonal with the highest accuracy as the coordinate calculation group, and determine the accuracy of the two diagonals in the coordinate calculation group: When only one diagonal angle in the coordinate calculation group has an accuracy greater than the accuracy threshold, the diagonal angle is selected as the reference angle, and the diagonal coordinates of another diagonal angle in the coordinate calculation group are calculated based on the reference angle; when there is no situation in the coordinate calculation group where the accuracy is greater than the accuracy threshold, the image processing unit re-denoises the acquired fixture image and calculates the diagonal coordinates.

6. The fixture grasping system for a mechanical claw according to claim 5, characterized in that: When calculating the accuracy, include: The ideal contour and the actual contour when the mechanical claw grasps the fixture are obtained, and the Hausdorff distance between the ideal contour and the actual contour is calculated. The accuracy has the following relationship: ； in, For accuracy, represents the actual contour, represents the ideal contour, Represents the Hausdorff distance between the ideal contour and the actual contour.

7. The fixture grasping system for a mechanical claw according to claim 6, characterized in that: When the calculation unit determines the coordinate threshold and the force threshold when the mechanical claw grasps normally according to the historical data set, it includes: Obtain the pre-input standard diagonal coordinates and the standard force data of each claw of the mechanical claw, and the horizontal and vertical coordinates of the coordinate threshold are in the following range: ； ； The stress threshold is in the following range: ； in, is the mean of all horizontal coordinates of the same diagonal in the historical data set, is the mean of all ordinates of the same diagonal in the historical data set, is the standard deviation of all horizontal coordinates of the same diagonal in the historical data set, is the standard deviation of all ordinates of the same diagonal in the historical data set, is the mean of all force data of the i-th mechanical gripper finger in the historical data set, is the standard deviation of all force data of the i-th mechanical gripper finger in the historical data set, k represents the tolerance range, 1≤k≤3.

8. The fixture grasping system for a mechanical claw according to claim 7, characterized in that: When judging whether there is an abnormality in the gripping of the mechanical claw according to the coordinate threshold and the force threshold, it includes: Extracting the diagonal coordinates and the force data of the claw finger corresponding to the monitoring time from the two-dimensional data matrix; judging whether the current diagonal coordinates are within the range of the coordinate threshold, if the abscissa and / or ordinate exceeds the range of the coordinate threshold, recording the monitoring time and the corresponding diagonal coordinates and marking them as abnormal points; The force data of each claw finger is detected to determine whether the force data is within the range of the force threshold. If the force data of any claw finger exceeds the range of the force threshold, the monitoring time and the corresponding force data are recorded as abnormal points.

9. The fixture grasping system for a mechanical claw according to claim 8, characterized in that: The early warning module is further configured to count the number of abnormal points and issue graded alarms according to the number of abnormal points.

10. The fixture grasping system for a mechanical claw according to claim 9, characterized in that: The image processing unit sets a first upper limit of the number of times of reacquiring the jig image and a second upper limit of the number of times of re-performing denoising and calculating the diagonal coordinates; When the number of times the jig image is reacquired meets the first number upper limit or when the number of times the denoising process is re-performed and the diagonal coordinates are calculated meets the preset second number upper limit, a warning signal is sent to the warning module.

Images