Lifting appliance detection method and system guided by mechanical arm

Abstract

The invention relates to the technical field of lifting appliance detection, and provides a lifting appliance detection method and system guided by a mechanical arm, and the lifting appliance detection method comprises the following steps: S1, selecting a reference point, a first detection point, a second detection point and a third detection point in a lifting appliance according to a preset detection requirement; s2, acquiring point clouds at the reference point, the first detection point, the second detection point and the third detection point; s3, on the basis of the point cloud characteristics of the reference point, the first detection point, the second detection point and the third detection point, at least two of an RANSAC segmentation algorithm, an Euclidean clustering segmentation algorithm and an edge detection algorithm are used to calculate the centroid coordinates of the point clouds at the corresponding positions; and S4, taking the reference point as a reference point, and obtaining error values of the first detection point, the second detection point and the third detection point according to the centroid coordinates of the point cloud.

Description

Technical Field

[0001] The present invention relates to the technical field of sling detection, and in particular to a sling detection method and system guided by a mechanical arm. Background Art

[0002] In modern industrial production, the quality control and inspection of slings have become particularly important. Slings usually have irregular shapes and complex surface features. Traditional manual or traditional inspection methods (such as contact three-dimensional coordinate measuring machines, laser scanning, etc.) face many challenges under such complex conditions, such as: Low inspection efficiency: Traditional methods often require a lot of manual intervention, and for complex slings, it takes a long time to complete comprehensive inspection; it is impossible to achieve multi-angle and all-round inspection, and for large-sized and complex-shaped slings, the limited viewing angle may result in some areas not being fully covered.

[0003] The common three-dimensional coordinate measuring machine (CMM) obtains three-dimensional coordinates by contacting the surface of the hanger with the probe. It is suitable for most simple workpieces, but it has many limitations for large, complex, and irregularly shaped workpieces: slow speed, especially when fine measurement is required, it takes a long time; poor flexibility, and it is difficult to perform all-round inspection of complex-shaped workpieces.

[0004] Therefore, a detection method based on robot arm guidance is needed to detect the spreader. Summary of the invention

[0005] In view of the above-mentioned defects, the purpose of the present invention is to propose a method and system for detecting hangers guided by a robot arm, aiming to improve the detection flexibility and coverage, realize efficient and automated detection, reduce damage to the workpiece surface, and improve detection accuracy.

[0006] To achieve this object, the present invention adopts the following technical solutions:

[0007] A method for detecting a spreader guided by a mechanical arm, wherein the mechanical arm is equipped with a three-dimensional structured light camera, and the mechanical arm is used to move so that the camera completes the detection according to the detection path. The method for detecting a spreader comprises the following steps:

[0008] S1: selecting a reference point, a first detection point, a second detection point and a third detection point in the spreader according to preset detection requirements;

[0009] S2: the camera acquires point clouds at the reference point, the first detection point, the second detection point and the third detection point;

[0010] S3: based on the point cloud characteristics at the reference point, the first detection point, the second detection point and the third detection point, respectively, at least two of the RANSAC segmentation algorithm, the Euclidean clustering segmentation algorithm and the edge detection algorithm are used to calculate the centroid coordinates of the point cloud at the corresponding position;

[0011] S4: Taking the reference point as a reference point, obtaining error values ​​of the first detection point, the second detection point, and the third detection point according to the centroid coordinates of the point cloud.

[0012] Preferably, step S3 includes:

[0013] Based on the point cloud characteristics at the reference point, RANSAC segmentation is first performed on the point cloud at the reference point to obtain the first point cloud after removing the reference plane;

[0014] The first point cloud is segmented according to the Euclidean clustering segmentation algorithm, and the cluster point cloud with the largest center value Z value is selected as the second point cloud;

[0015] Perform RANSAC segmentation on the second point cloud to remove noise and identify the centroid, and obtain the centroid coordinates of the point cloud at the reference point;

[0016] The point cloud characteristics at the reference point include that the reference point is connected to the plane below it.

[0017] Preferably, step S3 includes:

[0018] Based on the point cloud characteristics at the first detection point, the Euclidean clustering segmentation algorithm processes the point cloud at the first detection point to obtain a clustered point cloud;

[0019] The center value of each cluster point cloud is calculated, and the RANSAC segmentation algorithm processes the cluster point cloud with the largest center value Z value to identify the centroid coordinates of the first detection point;

[0020] The point cloud characteristics at the first detection point include that the plane where the first detection point is located is connected to the impurity plane, resulting in the plane where the first detection point is located being uneven.

[0021] Preferably, step S3 includes:

[0022] Based on the point cloud characteristics at the second detection point, Euclidean clustering segmentation, RANSAC segmentation, edge detection and Euclidean clustering segmentation are performed on the point cloud at the second detection point in sequence to obtain an outer ring point cloud and an inner ring point cloud;

[0023] Select the inner ring point cloud to calculate the centroid coordinates of the point cloud at the second detection point;

[0024] The point cloud characteristics at the second detection point include that the point cloud at the second detection point has two planes and the center is empty.

[0025] Preferably, the point cloud characteristics at the third detection point include a plane existing behind the third detection point;

[0026] Step S3 includes:

[0027] Based on the point cloud characteristics at the third detection point, after performing Euclidean clustering segmentation and RANSAC segmentation on the point cloud at the third detection point, the centroid coordinates of the point cloud at the third detection point are calculated;

[0028] For the point cloud at the third detection point before performing Euclidean clustering segmentation and RANSAC segmentation, the RANSAC fitting plane model method of the original point cloud is used to obtain the equation of the rear plane at the third detection point;

[0029] According to the equation and the coordinates of the centroid of the point cloud at the third detection point, the distance from the centroid of the point cloud at the third detection point to the rear plane at the third detection point is calculated.

[0030] Preferably, when the RANSAC segmentation algorithm segments a plane, the points on the plane to be segmented satisfy the relationship:

[0031] ax+by+cz=d;

[0032] Among them, (x, y, z) represents the three-dimensional coordinates of any point on the plane, (a, b, c) represents the direction of the plane, and (a 2 +b 2 +c 2 )=1, d represents the distance from the point of the point cloud to the plane;

[0033] When converting the plane parameters into a matrix, the expression of the plane parameters to be measured is: ∏=[a,b,c,d] T , three points are randomly extracted from the point cloud data using the random sampling method, satisfying the relationship:

[0034]

[0035] Among them, P represents any point in the scene, ∏ represents a plane, and (x, y, z) represents the three-dimensional coordinates of any point on the plane. Represents the calculation parameters and satisfies:

[0036] Set the distance threshold δ0. If the distance from any point P in the scene to the plane ∏ satisfies d<δ0, the point is recorded as an inside point, otherwise it is set as an outside point.

[0037] Preferably, the Euclidean clustering segmentation algorithm includes:

[0038] For a point cloud dataset P, determine a query point P i , set the distance threshold r, where the calculation formula of the threshold r is:

[0039]

[0040] Among them, p i With p jRepresents two points in the point cloud, p ik With p jk Represents any point in the neighborhood of two points;

[0041] Find the n nearest neighbor points P through KD-Tree j (j=1,2,...,n), calculate the Euclidean distance d from n neighboring points to the query point according to the calculation formula of the threshold r j ; Compare distance d j With the distance threshold r, the points less than r are classified into class M. The segmentation is completed when the number of points in class M no longer increases.

[0042] Further, the point clouds at the reference point, the first detection point, the second detection point and the third detection point acquired in step S2 are all ROI point clouds obtained by filtering using straight-through filtering.

[0043] A mechanical arm guided sling detection system, using the sling detection method as described above, the sling detection system comprising:

[0044] A point selection module, used to select a reference point, a first detection point, a second detection point and a third detection point in the spreader according to preset detection requirements;

[0045] A point cloud acquisition module, used to acquire point clouds at a reference point, a first detection point, a second detection point, and a third detection point;

[0046] A coordinate calculation module, used to calculate the centroid coordinates of the point cloud at the corresponding position using at least two of a RANSAC segmentation algorithm, a Euclidean clustering segmentation algorithm, and an edge detection algorithm based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, respectively;

[0047] The error detection module is used to obtain error values ​​of the first detection point, the second detection point and the third detection point according to the centroid coordinates of the point cloud with the reference point as the reference point.

[0048] One of the above technical solutions has the following advantages or beneficial effects:

[0049] The present invention guides the structured light camera through a robotic arm, and the entire detection process is automated without human intervention, thereby greatly improving the detection speed and efficiency; the robotic arm can move quickly and automatically scan different parts of the sling, avoiding the blind angle problem of viewing angle in traditional methods, thereby improving the comprehensiveness and coverage of detection; non-contact three-dimensional structured light imaging is adopted, so the workpiece will not be directly contacted during the detection process, avoiding the problem of surface damage that may be caused by contact measurement; according to the characteristics of different detection points, the reasonable use of the RANSAC segmentation algorithm, the Euclidean clustering algorithm and the edge detection can help identify the centroid of the detection point of the sling, thereby improving the accuracy and efficiency of the three-dimensional detection of the sling. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0051] Figure 1 is a flow chart of a method for detecting a spreader guided by a robot arm provided in an embodiment of the present invention;

[0052] Figure 2 is a specific flow chart of a method for detecting a spreader guided by a robot arm provided in another embodiment of the present invention;

[0053] Figure 3 1 is a diagram showing the detection effect of different points of the method for detecting a sling guided by a robot arm provided in an embodiment of the present invention;

[0054] Figure 4 is a detection effect diagram of a hanger detection system guided by a mechanical arm provided in an embodiment of the present invention;

[0055] Figure 5 is an execution flow chart of a hanger detection system guided by a mechanical arm provided by an embodiment of the present invention;

[0056] Figure 6 It is a structural schematic diagram of a hanger detection system guided by a robot arm provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0057] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0058] In the present invention, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0059] In modern industrial production, the quality control and inspection of slings have become particularly important. Slings usually have irregular shapes and complex surface features. Traditional manual or traditional inspection methods (such as contact three-dimensional coordinate measuring machines, laser scanning, etc.) face many challenges under such complex conditions, such as: Low inspection efficiency: Traditional methods often require a lot of manual intervention, and for complex slings, it takes a long time to complete comprehensive inspection; it is impossible to achieve multi-angle and all-round inspection, and for large-sized and complex-shaped slings, the limited viewing angle may result in some areas not being fully covered.

[0060] The common three-dimensional coordinate measuring machine (CMM) obtains three-dimensional coordinates by contacting the surface of the hanger with the probe. It is suitable for most simple workpieces, but it has many limitations for large, complex, and irregularly shaped workpieces: slow speed, especially when fine measurement is required, it takes a long time; poor flexibility, and it is difficult to perform all-round inspection of complex-shaped workpieces.

[0061] Laser scanning measurement in traditional detection means is a non-contact measurement method. It obtains three-dimensional information by projecting a laser beam onto the surface of the workpiece and collecting the position of the reflected laser. Laser scanning technology is used for the shape measurement of complex workpieces and is greatly affected by the surface material: laser scanning is less effective for materials with smooth surfaces or strong reflectivity, and data loss or noise is prone to occur; viewing angle limitation: fixed laser scanners can usually only scan from one perspective. For workpieces with complex shapes, it is difficult to provide full coverage, resulting in data missing in some areas; highest cost: laser scanning equipment and data processing costs are high, especially in the detection of large workpieces, which requires equipment and a large amount of data processing resources.

[0062] Therefore, a detection method based on robot arm guidance is needed to detect the spreader.

[0063] like Figure 1 As shown, a preferred embodiment of the present invention is a method for detecting a spreader guided by a robot arm, wherein the robot arm is equipped with a non-contact three-dimensional structured light camera, and the robot arm is used to move so that the camera completes the detection according to the detection path. The method for detecting a spreader comprises the following steps:

[0064] S1: selecting a reference point, a first detection point, a second detection point and a third detection point in the spreader according to preset detection requirements;

[0065] S2: the camera acquires point clouds at the reference point, the first detection point, the second detection point and the third detection point;

[0066] S3: based on the point cloud characteristics at the reference point, the first detection point, the second detection point and the third detection point, respectively, at least two of the RANSAC segmentation algorithm, the Euclidean clustering segmentation algorithm and the edge detection algorithm are used to calculate the centroid coordinates of the point cloud at the corresponding position;

[0067] S4: Taking the reference point as a reference point, obtaining error values ​​of the first detection point, the second detection point, and the third detection point according to the centroid coordinates of the point cloud.

[0068] Before step S1, the OCR point can be detected first, and the robot arm moves to the OCR point position. Through cropping, filtering, and brightening, the PaddleOCR model is used for identification to obtain the number of the entire sling, which is convenient for identifying the sling to avoid confusion. In step S1, before the detection of the sling, it is necessary to first select multiple key points for subsequent point cloud data collection and processing according to the preset detection requirements. These key points include reference points and multiple detection points. The reference points are used as reference benchmarks in the subsequent measurement process, while other detection points are used for precise measurement and error calculation of specific parts of the sling. The reference point is a fixed point for reference, usually selected at a stable position of the sling as a benchmark for all subsequent measurements of all other points. The selection of the reference point is crucial and will affect the accuracy of the entire detection process. The first detection point, the second detection point, and the third detection point are points selected on the sling for specific measurements. Their positions and numbers depend on the design of the sling and the detection requirements (such as different slings have different purposes and functions, and different detection points are selected).

[0069] In step S2, in the detection guided by the robot arm, the robot arm drives the structured light camera to move, and the structured light camera scans the selected reference points and detection points to obtain the corresponding point cloud data, where the point cloud data is a collection of all points on the surface of the object, and the three-dimensional coordinates of these points can accurately describe the shape and structure of the object. Point cloud data is a collection of a series of points on the surface of an object captured by a three-dimensional sensor (such as a structured light camera), each point has three coordinates (x, y, z), which describes the spatial distribution of various positions on the surface of the object. A structured light camera is a sensor that projects a known light structure (such as stripes or grids) onto the surface of an object and captures the deformed image, thereby calculating the three-dimensional shape of the object. A structured light camera can provide high-precision point cloud data.

[0070] For the collected point cloud data, different algorithms are used for processing and analysis according to the characteristics of different points. The RANSAC segmentation algorithm, Euclidean clustering segmentation algorithm and edge detection algorithm can be used to separate noise points, fit planes, identify object features, etc. from the point cloud data. After obtaining the correct point cloud area, the centroid coordinates of the area are calculated. Among them, the RANSAC segmentation algorithm is an effective and robust estimation algorithm that divides data into subsets of inner points and outer points. The algorithm designs a model based on the problem as the segmentation standard, uses random sampling to continuously input different data from the total data set, continuously iterates and calculates model parameters, and then re-divides the original data set according to the model. The Euclidean clustering segmentation algorithm divides the point cloud into different clusters based on the distance between points. Euclidean clustering is suitable for situations where the surface distribution of objects is relatively uniform, and can effectively remove background or irrelevant point clouds. After the RANSAC algorithm is used to segment the planes in the scene, a large number of irrelevant environmental point clouds are removed. However, it is still impossible to determine how many objects to be tested exist in the scene, and which cluster each point in the space belongs to. The essence of Euclidean clustering is to classify points with similar distances into one category. Assuming there is a point in the point cloud, the Euclidean distance is defined as the degree of closeness between two points, and the distance between neighboring points is used as the standard to achieve point cloud clustering segmentation. The edge detection algorithm detects the edges of objects in the point cloud, which can help identify the contours and appearance features of the object. The center of mass is the geometric center of an object or part of an object. The center of mass coordinates are used to describe the position of the part of the object, and are usually obtained by calculating the average coordinates of the point cloud.

[0071] During the inspection process, by calculating the centroid coordinates of different inspection points, the coordinates of the reference point can be used as a reference to calculate the error values ​​of other inspection points. Through error value analysis, the accuracy of the spreader at each position can be evaluated to determine whether it meets the design requirements, such as Figure 2 2 is a specific flow chart of this embodiment, wherein detection points 1 and 5 are the first detection points, detection points 2 and 4 are the second detection points, and detection point 3 is the third detection point.

[0072] The present invention aims at the low efficiency, limited flexibility and fragility of the traditional three-coordinate contact measurement method. The probe needs to contact the surface of the workpiece, which is easy to damage the workpiece surface or the probe itself. Especially when detecting fragile or precise workpieces, the laser scanning measurement method is greatly affected by the surface material, has a limited viewing angle and high cost. A method for detecting a sling guided by a robotic arm is proposed. By guiding the structured light camera by the robotic arm, the whole detection process is automated without manual intervention, thereby greatly improving the detection speed and efficiency. The robotic arm can move quickly and automatically scan different parts of the sling, avoiding the blind angle problem of the viewing angle in the traditional method, thereby improving the comprehensiveness and coverage of the detection. Non-contact three-dimensional structured light imaging is adopted, so the workpiece is not directly contacted during the detection process, thereby avoiding the surface damage problem that may be caused by contact measurement. According to the characteristics of different detection points, the reasonable use of the RANSAC segmentation algorithm, the Euclidean clustering algorithm and the edge detection can help identify the centroid of the detection point of the sling, thereby improving the accuracy and efficiency of the three-dimensional detection of the sling.

[0073] Preferably, step S3 includes:

[0074] Based on the point cloud characteristics at the reference point, RANSAC segmentation is first performed on the point cloud at the reference point to obtain the first point cloud after removing the reference plane;

[0075] The first point cloud is segmented according to the Euclidean clustering segmentation algorithm, and the cluster point cloud with the largest center value Z value is selected as the second point cloud;

[0076] Perform RANSAC segmentation on the second point cloud to remove noise and identify the centroid, and obtain the centroid coordinates of the point cloud at the reference point;

[0077] The point cloud characteristics at the reference point include that the reference point is connected to the plane below it.

[0078] like Figure 3 As shown, the reference point is Figure 3The points in the first row of are connected to the lower plane (yellow area) and serve as reference points for calculating the coordinate deviations of other detection points. The Z value represents the height value of each point in the point cloud, corresponding to the Z-axis coordinate in three-dimensional space. In this step, the clustered point cloud with the largest Z value is selected as the second point cloud. Since the reference point is connected to the lower plane, if the Euclidean clustering segmentation is used directly on the reference point, under-segmentation will occur. Therefore, when processing point cloud data, the RANSAC algorithm is first used to identify and remove the reference plane part. By randomly selecting some points and fitting the plane, the RANSAC algorithm can effectively find data points consistent with the reference plane and remove these points, thereby removing the ground interference connected to the reference point (the first point cloud), creating a clean point cloud data foundation for subsequent target object extraction. After the reference plane is removed, the remaining point cloud is segmented using the Euclidean clustering algorithm. By setting a distance threshold, the algorithm classifies points close to each other into one category, and selects the cluster with the largest Z value (height) as the target point cloud (second point cloud). This can effectively identify the area where the target object is located, reduce the influence of background noise and irrelevant points, and ensure the accuracy of subsequent processing. For the selected target point cloud, the RANSAC algorithm is further used for fine segmentation to remove the noise points and optimize the quality of the point cloud by fitting a more accurate model. After that, the centroid of the target point cloud is calculated to obtain its three-dimensional coordinates. This centroid position represents the position of the target object and is compared with the coordinates of the reference point to evaluate the error and ensure the accuracy of positioning.

[0079] Preferably, step S3 includes:

[0080] Based on the point cloud characteristics at the first detection point, the Euclidean clustering segmentation algorithm processes the point cloud at the first detection point to obtain a clustered point cloud;

[0081] The center value of each cluster point cloud is calculated, and the RANSAC segmentation algorithm processes the cluster point cloud with the largest center value Z value to identify the centroid coordinates of the first detection point;

[0082] The point cloud characteristics at the first detection point include that the plane where the first detection point is located is connected to the impurity plane, resulting in the plane where the first detection point is located being uneven.

[0083] like Figure 3As shown in the second row, it represents the calculation process of the centroid of the first detection point. The first detection point is above, and the impurity plane is below. It can be seen that the target plane above is not very flat. The Euclidean clustering segmentation algorithm divides the point cloud data into multiple clusters by setting a distance threshold. Each cluster contains points with similar spatial positions. The goal of this process is to extract interrelated point groups from complex point clouds to represent an object or an area. By calculating the distance between points, Euclidean clustering can effectively distinguish the points of the target object from the background noise, thereby laying the foundation for subsequent accurate segmentation. After completing the Euclidean clustering, the next step is to calculate the center value of each cluster. The center value is obtained by calculating the coordinate mean of all points in the cluster, which represents the geometric center of the cluster. In this step, special attention is paid to the cluster with the largest Z value (vertical coordinate), because the target object is usually located at a higher position and its Z value is larger. In this way, the cluster that is most likely to represent the target object can be determined, thereby preparing for the next step of RANSAC segmentation. After selecting the cluster with the largest Z value, the RANSAC algorithm is applied to further fine segment the cluster point cloud. RANSAC removes points that do not conform to the plane characteristics by randomly selecting points and fitting the plane model, retaining the point cloud data that truly belongs to the target plane. The key is to accurately extract the plane of the target object by removing noise and impurities, especially when the plane of the target object is connected to the impurity plane and is not flat. RANSAC can significantly improve the segmentation accuracy and ensure that the target point cloud is accurately extracted. In the target point cloud obtained by RANSAC segmentation, the last step is to calculate the center of mass. The center of mass is determined by calculating the mean coordinates of all points in the point cloud to represent the position of the object. The calculation of the center of mass provides accurate three-dimensional coordinates for the target object, helping to determine the exact position of the target object in space. It ensures that the point cloud data obtained through the previous segmentation and extraction steps can be converted into specific spatial coordinates, thereby supporting subsequent detection.

[0084] Preferably, step S3 includes:

[0085] Based on the point cloud characteristics at the second detection point, Euclidean clustering segmentation, RANSAC segmentation, edge detection and Euclidean clustering segmentation are performed on the point cloud at the second detection point in sequence to obtain an outer ring point cloud and an inner ring point cloud;

[0086] Select the inner ring point cloud to calculate the centroid coordinates of the point cloud at the second detection point;

[0087] The point cloud characteristics at the second detection point include that the point cloud at the second detection point has two planes and the center is empty.

[0088] like Figure 3As shown in the figure, the point cloud at the second detection point will have two planes, and the center of the detection point is empty; Euclidean clustering rough segmentation is based on the Euclidean distance between points to divide the clusters. By setting a larger distance threshold, the point cloud is preliminarily grouped. Each cluster represents a potential object or area. The Euclidean clustering algorithm will classify points with closer distances into the same cluster, and points with farther distances into other clusters. The purpose of rough segmentation is to preliminarily separate different areas (outer ring and inner ring) in the point cloud from the noise data. At this stage, the accuracy of segmentation is not required to be high. The obvious different parts in the point cloud are extracted, and some discrete noise points are excluded to lay the foundation for subsequent processing. Specifically, when performing Euclidean clustering, the algorithm calculates the similarity between each pair of points based on the distance between the points, and determines whether to classify them into the same cluster based on the set threshold. In this way, the clustering algorithm divides the point cloud into multiple clusters, each cluster represents a larger area, which may contain multiple objects or different parts of an object.

[0089] RANSAC plane segmentation randomly selects points and fits the plane model to screen out the point cloud that meets the plane hypothesis, and then extracts the plane structure from the complex point cloud data. It can find the point group that meets the model in the case of more noise or complex data. RANSAC helps to eliminate points that do not meet the plane characteristics and determine the plane area around the second detection point. It is especially suitable for extracting target planes in multiple planes or complex backgrounds. Since the point cloud at the second detection point will have two planes, and the center of the detection point is empty, edge detection is required after segmentation. The purpose of edge detection is to identify the area where the surface changes sharply in the point cloud data, so as to extract the edge information of the object. In point cloud processing, the edge position can be found by calculating the normal or curvature of the point. The place where the normal changes greatly is often the outline of the object. Edge detection helps to further extract the boundary of the target area from the segmented plane, especially in complex scenes, and can clearly separate the inner and outer ring parts to avoid confusion and omissions.

[0090] Euclidean clustering fine segmentation is based on coarse segmentation, and further refines the segmentation of point clouds through more accurate clustering algorithms. By adjusting the distance threshold or other clustering parameters, fine segmentation can divide the target area at a finer granularity, ensuring the clear separation of the inner and outer rings. It is suitable for scenes with high point cloud density or complex boundaries. By refining the segmentation results, the accurate target area is finally extracted to reduce the possibility of misclassification. The centroid calculation is to average the coordinates of all point clouds in the target area to obtain the geometric center position of the area. By calculating the centroid of the inner ring point cloud, the three-dimensional coordinates of the second detection point can be obtained as the representative position of the target area. The centroid is used as the basis for positioning the object. In subsequent processing or analysis, it provides an accurate reference point, which can provide key data for subsequent target recognition or positioning tasks, ensuring that the system can correctly identify and process the target area.

[0091] Preferably, the point cloud characteristics at the third detection point include a plane existing behind the third detection point;

[0092] Step S3 includes:

[0093] Based on the point cloud characteristics at the third detection point, after performing Euclidean clustering segmentation and RANSAC segmentation on the point cloud at the third detection point, the centroid coordinates of the point cloud at the third detection point are calculated;

[0094] For the point cloud at the third detection point before performing Euclidean clustering segmentation and RANSAC segmentation, the RANSAC fitting plane model method of the original point cloud is used to obtain the equation of the rear plane at the third detection point;

[0095] According to the equation and the coordinates of the centroid of the point cloud at the third detection point, the distance from the centroid of the point cloud at the third detection point to the rear plane at the third detection point is calculated.

[0096] The plane is fitted using the RANSAC algorithm. The plane model is fitted by randomly selecting a subset of the point cloud, and the matching degree between the model and the data is evaluated, and finally the equation of the rear plane is extracted. The centroid calculation is achieved by finding the geometric center of the target point cloud area, that is, the average value of the coordinates of all points in the area. For the third detection point, the centroid is first calculated in the precise point cloud area obtained after Euclidean clustering segmentation and RANSAC segmentation, and the original point cloud data is fitted with the RANSAC method to obtain the mathematical equation of the rear plane. The RANSAC algorithm randomly selects points from the data and calculates the plane model, and continuously optimizes it to finally obtain a plane equation that best fits most point cloud data. The distance calculation from the centroid to the rear plane is based on the fitted plane equation and the calculated centroid coordinates, using the distance formula between the plane equation and the point. By solving the vertical distance between the centroid coordinates and the plane equation, the shortest distance between the centroid and the plane can be obtained. This step is used to determine the relative position change between the target point cloud and the rear plane, and to monitor whether the point cloud has shifted. This distance value can be used as a measure of displacement to help evaluate whether the point cloud changes in the system are as expected, which is beneficial for detection.

[0097] Preferably, when the RANSAC segmentation algorithm segments a plane, the points on the plane to be segmented satisfy the relationship:

[0098] ax+by+cz=d;

[0099] Among them, (x, y, z) represents the three-dimensional coordinates of any point on the plane, (a, b, c) represents the direction of the plane, and (a 2 +b 2 +c 2 )=1, d represents the distance from the point of the point cloud to the plane;

[0100] When converting the plane parameters into a matrix, the expression of the plane parameters to be measured is: ∏=[a,b,c,d] T , three points are randomly extracted from the point cloud data using the random sampling method, satisfying the relationship:

[0101]

[0102] Among them, P represents any point in the scene, ∏ represents a plane, and (x, y, z) represents the three-dimensional coordinates of any point on the plane. Represents the calculation parameters and satisfies:

[0103] Set the distance threshold δ0. If the distance from any point P in the scene to the plane ∏ satisfies d<δ0, the point is recorded as an inside point, otherwise it is set as an outside point.

[0104] Specifically, RANSAC randomly selects a minimum data set to calculate the model parameters, then evaluates the degree of conformity of all data points with the model, calculates the number of inliers and outliers, and the RANSAC algorithm selects the model with the most inliers as the final result through multiple iterations, thereby effectively eliminating outliers and finding the best fitting model.

[0105] Preferably, the Euclidean clustering segmentation algorithm includes:

[0106] For a point cloud dataset P, determine a query point P i , set the distance threshold r, where the calculation formula of the threshold r is:

[0107]

[0108] Among them, p i With p j Represents two points in the point cloud, p ik With p jk Represents any point in the neighborhood of two points;

[0109] Find the n nearest neighbor points P through KD-Tree j (j=1,2,...,n), calculate the Euclidean distance d from n neighboring points to the query point according to the calculation formula of the threshold r j ; Compare distance d j With the distance threshold r, the points less than r are classified into class M. The segmentation is completed when the number of points in class M no longer increases.

[0110] Specifically, first select a point as the query point and set a distance threshold. Only other points that are close to the query point will be classified into the same category. By calculating the Euclidean distance between the query point and other points, it can be determined which points belong to the same cluster. In order to improve efficiency, the algorithm uses data structures such as KD-Tree to speed up the search for neighboring points and avoid calculating all points one by one. Then the algorithm continuously expands the current cluster and gradually adds qualified points in the neighborhood to the current cluster until the cluster can no longer be expanded. Finally, all points will be classified into different clusters to complete the segmentation of the point cloud.

[0111] Further, the point clouds at the reference point, the first detection point, the second detection point and the third detection point acquired in step S2 are all ROI point clouds obtained by filtering using straight-through filtering.

[0112] The point clouds at the reference point, the first detection point, the second detection point, and the third detection point are all ROI (Region of Interest) point clouds extracted from the original point cloud data by the pass-through filter method. Pass-through filtering is a commonly used point cloud data processing technology. It filters out point cloud data within a specified area by setting a minimum and maximum filter range in a specific coordinate axis direction. In this way, only the point cloud within the region of interest is retained, and irrelevant background or noise is removed, thereby improving the accuracy and efficiency of subsequent processing. The reference point and the detection point represent the key positions or feature points in the region for further detection.

[0113] like Figure 6 , a mechanical arm guided spreader detection system, applied to the spreader detection method as described above, characterized in that the spreader detection system comprises:

[0114] A point selection module, used to select a reference point, a first detection point, a second detection point and a third detection point in the spreader according to preset detection requirements;

[0115] A point cloud acquisition module, used to acquire point clouds at a reference point, a first detection point, a second detection point, and a third detection point;

[0116] A coordinate calculation module, used to calculate the centroid coordinates of the point cloud at the corresponding position using at least two of a RANSAC segmentation algorithm, a Euclidean clustering segmentation algorithm, and an edge detection algorithm based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, respectively;

[0117] The error detection module is used to obtain error values ​​of the first detection point, the second detection point and the third detection point according to the centroid coordinates of the point cloud with the reference point as the reference point.

[0118] Figure 4 This is the detection effect diagram of the spreader detection system guided by the robot arm. Figure 5 This is a flow chart of an embodiment of a sling detection system guided by a robot arm. This embodiment implements a sling detection method and implementation process. Please refer to the above embodiments and will not be described one by one here.

[0119] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0120] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for detecting a spreader guided by a robot arm, characterized in that: The mechanical arm is equipped with a three-dimensional structured light camera, and the mechanical arm is used to move so that the camera completes the detection according to the detection path. The hanger detection method includes the following steps: S1: selecting a reference point, a first detection point, a second detection point and a third detection point in the spreader according to preset detection requirements; S2: the camera acquires point clouds at the reference point, the first detection point, the second detection point and the third detection point; S3: based on the point cloud characteristics at the reference point, the first detection point, the second detection point and the third detection point, respectively, at least two of the RANSAC segmentation algorithm, the Euclidean clustering segmentation algorithm and the edge detection algorithm are used to calculate the centroid coordinates of the point cloud at the corresponding position; S4: Taking the reference point as a reference point, obtaining error values ​​of the first detection point, the second detection point, and the third detection point according to the centroid coordinates of the point cloud.

2. The method for detecting a sling according to claim 1, characterized in that: Step S3 includes: Based on the point cloud characteristics at the reference point, RANSAC segmentation is first performed on the point cloud at the reference point to obtain the first point cloud after removing the reference plane; The first point cloud is segmented according to the Euclidean clustering segmentation algorithm, and the cluster point cloud with the largest center value Z value is selected as the second point cloud; Perform RANSAC segmentation on the second point cloud to remove noise and identify the centroid, and obtain the centroid coordinates of the point cloud at the reference point; The point cloud characteristics at the reference point include that the reference point is connected to the plane below it.

3. The method for detecting a sling according to claim 1, characterized in that: Step S3 includes: Based on the point cloud characteristics at the first detection point, the Euclidean clustering segmentation algorithm processes the point cloud at the first detection point to obtain a clustered point cloud; The center value of each cluster point cloud is calculated, and the RANSAC segmentation algorithm processes the cluster point cloud with the largest center value Z value to identify the centroid coordinates of the first detection point; The point cloud characteristics at the first detection point include that the plane where the first detection point is located is connected to the impurity plane, resulting in the plane where the first detection point is located being uneven.

4. The method for detecting a sling according to claim 1, characterized in that: Step S3 includes: Based on the point cloud characteristics at the second detection point, perform Euclidean clustering segmentation, RANSAC segmentation, edge detection and Euclidean clustering segmentation on the point cloud at the second detection point in sequence to obtain an outer ring point cloud and an inner ring point cloud; Select the inner ring point cloud to calculate the centroid coordinates of the point cloud at the second detection point; The point cloud characteristics at the second detection point include that the point cloud at the second detection point has two planes and the center is empty.

5. The method for detecting a sling according to claim 1, characterized in that: The point cloud characteristics at the third detection point include a plane existing behind the third detection point; Step S3 includes: Based on the point cloud characteristics at the third detection point, after performing Euclidean clustering segmentation and RANSAC segmentation on the point cloud at the third detection point, the centroid coordinates of the point cloud at the third detection point are calculated; For the point cloud at the third detection point before performing Euclidean clustering segmentation and RANSAC segmentation, the RANSAC fitting plane model method of the original point cloud is used to obtain the equation of the rear plane at the third detection point; According to the equation and the coordinates of the centroid of the point cloud at the third detection point, the distance from the centroid of the point cloud at the third detection point to the rear plane at the third detection point is calculated.

6. The method for detecting a sling according to claim 1, characterized in that: When the RANSAC segmentation algorithm segments a plane, the points on the plane to be segmented satisfy the relationship: ax+by+cz=d; Among them, (x, y, z) represents the three-dimensional coordinates of any point on the plane, (a, b, c) represents the direction of the plane, and (a 2 +b 2 +c 2 )=1, d represents the distance from the point of the point cloud to the plane; When converting the plane parameters into a matrix, the expression of the plane parameters to be measured is: ∏=[a,b,c,d] T , three points are randomly extracted from the point cloud data using the random sampling method, satisfying the relationship: Among them, P represents any point in the scene, ∏ represents a plane, and (x, y, z) represents the three-dimensional coordinates of any point on the plane. Represents the calculation parameters and satisfies: Set the distance threshold δ0. If the distance from any point P in the scene to the plane ∏ satisfies d<δ0, the point is recorded as an inside point, otherwise it is set as an outside point.

7. The method for detecting a sling according to claim 1, characterized in that: The Euclidean clustering segmentation algorithm includes: For a point cloud dataset P, determine a query point P i , set the distance threshold r, where the calculation formula of the threshold r is: Among them, p i With p j Represents two points in the point cloud, p ik With p jk Represents any point in the neighborhood of two points; Find the n nearest neighbor points P through KD-Tree j (j=1,2,...,n), calculate the Euclidean distance d from n neighboring points to the query point according to the calculation formula of the threshold r j ; Compare distance d j With the distance threshold r, the points less than r are classified into class M. The segmentation is completed when the number of points in class M no longer increases.

8. The method for detecting a sling according to any one of claims 1 to 7, characterized in that: The point clouds at the reference point, the first detection point, the second detection point and the third detection point acquired in step S2 are all ROI point clouds obtained by filtering using a straight-through filter.

9. A mechanical arm guided spreader detection system, applied to the spreader detection method according to any one of claims 1 to 8, characterized in that: The spreader detection system comprises: A point selection module, used to select a reference point, a first detection point, a second detection point and a third detection point in the spreader according to preset detection requirements; A point cloud acquisition module, used to acquire point clouds at a reference point, a first detection point, a second detection point, and a third detection point; A coordinate calculation module, used to calculate the centroid coordinates of the point cloud at the corresponding position using at least two of a RANSAC segmentation algorithm, a Euclidean clustering segmentation algorithm, and an edge detection algorithm based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, respectively; The error detection module is used to obtain error values ​​of the first detection point, the second detection point and the third detection point according to the centroid coordinates of the point cloud with the reference point as the reference point.

Images