A crane guide system and method

Abstract

The present application relates to the technical field of sling detection, and proposes a kind of mechanical arm guided sling detection method and system, the sling detection method includes the following steps: S1: according to the preset detection requirement in the sling selects reference point, first detection point, second detection point and third detection point;S2: obtain the point cloud at reference point, first detection point, second detection point and third detection point;S3: respectively based on the point cloud characteristics at reference point, first detection point, second detection point and third detection point, at least two of using RANSAC segmentation algorithm, Euclidean clustering segmentation algorithm and edge detection algorithm calculate the centroid coordinates of corresponding position point cloud;S4: with reference point as reference point, according to the centroid coordinates of the point cloud, obtain the error value of first detection point, second detection point and third detection point.

Description

Technical Field

[0001] This invention relates to the field of lifting equipment inspection technology, and in particular to a lifting equipment inspection method and system guided by a robotic arm. Background Technology

[0002] In modern industrial production, the quality control and inspection of lifting equipment has become particularly important. Lifting equipment usually has irregular shapes and complex surface features. Traditional manual or traditional inspection methods (such as contact coordinate measuring machines, laser scanning, etc.) face many challenges under such complex conditions, such as: low inspection efficiency: traditional methods often require a lot of manual intervention, and for complex lifting equipment, it takes a long time to complete a comprehensive inspection; it is impossible to achieve multi-angle and all-round inspection, and for large-sized and complex-shaped lifting equipment, the limited field of view may result in some areas not being fully covered.

[0003] Common coordinate measuring machines (CMMs) acquire three-dimensional coordinates by having a probe contact the surface of a hanger. They are suitable for most simple workpieces, but they have many limitations for large, complex, and irregularly shaped workpieces: they are relatively slow, especially when fine measurements are required, and they are not flexible enough to perform all-round inspections of complex-shaped workpieces.

[0004] Therefore, a detection method based on robotic arm guidance is needed to detect lifting devices. Summary of the Invention

[0005] To address the aforementioned shortcomings, the present invention aims to propose a robotic arm-guided lifting fixture inspection method and system, which aims to improve inspection flexibility and coverage, achieve efficient automated inspection, reduce damage to workpiece surfaces, and improve inspection accuracy.

[0006] To achieve this objective, the present invention adopts the following technical solution:

[0007] A method for inspecting a lifting device guided by a robotic arm, wherein the robotic arm is equipped with a three-dimensional structured light camera, and the robotic arm is used to move so that the camera completes the inspection according to a detection path. The method for inspecting the lifting device includes the following steps:

[0008] S1: Select a reference point, a first detection point, a second detection point, and a third detection point in the lifting device according to the preset detection requirements;

[0009] S2: The camera acquires point clouds at the reference point, the first detection point, the second detection point, and the third detection point;

[0010] S3: Based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, calculate the centroid coordinates of the point cloud at the corresponding locations using at least two of the RANSAC segmentation algorithm, Euclidean clustering segmentation algorithm, and edge detection algorithm.

[0011] S4: Using the reference point as the reference point, obtain the error values ​​of the first detection point, the second detection point, and the third detection point based on the centroid coordinates of the point cloud.

[0012] Preferably, step S3 includes:

[0013] Based on the point cloud characteristics at the reference point, RANSAC segmentation is first performed on the point cloud at the reference point to obtain the first point cloud after removing the reference plane.

[0014] The first point cloud is segmented according to the Euclidean clustering segmentation algorithm, and the cluster point cloud with the largest center value Z value is selected as the second point cloud;

[0015] Perform RANSAC segmentation on the second point cloud to remove noise and identify the centroid, and obtain the centroid coordinates of the point cloud at the reference point;

[0016] The point cloud characteristics at the reference point include the connection between the reference point and its lower plane.

[0017] Preferably, step S3 includes:

[0018] Based on the point cloud characteristics at the first detection point, the Euclidean clustering segmentation algorithm processes the point cloud at the first detection point to obtain a clustered point cloud;

[0019] The centroid of each cluster point cloud is calculated. The RANSAC segmentation algorithm processes the cluster point cloud with the largest centroid Z value and then identifies the centroid coordinates at the first detection point.

[0020] Among them, the point cloud characteristics at the first detection point include the fact that the plane where the first detection point is located is connected to the impurity plane, resulting in the plane where the first detection point is located being uneven.

[0021] Preferably, step S3 includes:

[0022] Based on the point cloud characteristics at the second detection point, Euclidean clustering segmentation, RANSAC segmentation, edge detection, and Euclidean clustering segmentation are sequentially performed on the point cloud at the second detection point to obtain the outer ring point cloud and the inner ring point cloud.

[0023] Select the centroid coordinates of the point cloud at the second detection point by computing the inner ring point cloud;

[0024] Among them, the point cloud characteristics at the second detection point include the existence of two planes in the point cloud at the second detection point with an empty center.

[0025] Preferably, the point cloud characteristics at the third detection point include the existence of a plane behind the third detection point;

[0026] Step S3 includes:

[0027] Based on the point cloud characteristics at the third detection point, Euclidean clustering and RANSAC segmentation are performed on the point cloud at the third detection point, and then the centroid coordinates of the point cloud at the third detection point are calculated.

[0028] The equation of the back plane at the third detection point is obtained by using the RANSAC fitting plane model method of the original point cloud before performing Euclidean clustering segmentation and RANSAC segmentation.

[0029] Based on the equation and the centroid coordinates of the point cloud at the third detection point, the distance from the centroid of the point cloud at the third detection point to the plane behind the third detection point is calculated.

[0030] Preferably, when the RANSAC segmentation algorithm segments a plane, the points on the plane to be segmented satisfy the following relationship:

[0031] ax + by + cz = d;

[0032] Where (x,y,z) represents the three-dimensional coordinates of any point in the plane, and (a,b,c) represents the direction of the plane, satisfying (a 2 +b 2 +c 2 ) = 1, where d represents the distance from the point cloud location to the plane;

[0033] When converting planar parameters into a matrix, let the expression for the parameters of the plane to be measured be: ∏=[a,b,c,d] T Three points are randomly extracted from the point cloud data using a random sampling method, satisfying the following relationship:

[0034]

[0035] Where P represents any point in the scene, ∏ represents a plane, and (x, y, z) represents the three-dimensional coordinates of any point on the plane. Describe the calculated parameters and satisfy:

[0036] Set a distance threshold δ0. If the distance from any point P in the scene to the plane ∏ satisfies d < δ0, the point is recorded as an in-place point; otherwise, it is recorded as an out-of-place point.

[0037] Preferably, the Euclidean clustering segmentation algorithm includes:

[0038] For a point cloud dataset P, determine a query point P. i Set a distance threshold r, where the formula for calculating the threshold r is:

[0039]

[0040] Where, p i With p jLet p represent two points in a point cloud. ik With p jk This represents any point in the neighborhood of two points;

[0041] Find the n nearest neighbors P of the KD-Tree. j (j=1,2,...,n), calculate the Euclidean distance d from the n nearest neighbors to the query point according to the formula for calculating the threshold r. j Compare distance d j Points with a distance threshold r are assigned to class M, and the segmentation is completed when the number of points in class M no longer increases.

[0042] Furthermore, the point clouds at the reference point, the first detection point, the second detection point, and the third detection point obtained in step S2 are all ROI point clouds obtained by using a pass-through filter.

[0043] A robotic arm-guided lifting device inspection system, employing the lifting device inspection method described above, the lifting device inspection system comprising:

[0044] The point selection module is used to select a reference point, a first detection point, a second detection point, and a third detection point in the lifting device according to preset detection requirements;

[0045] The point cloud acquisition module is used to acquire point clouds at the reference point, the first detection point, the second detection point, and the third detection point;

[0046] The coordinate calculation module is used to calculate the centroid coordinates of the point cloud at the corresponding location based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, using at least two of the RANSAC segmentation algorithm, the Euclidean clustering segmentation algorithm, and the edge detection algorithm.

[0047] The error detection module is used to obtain the error values ​​of the first detection point, the second detection point, and the third detection point based on the centroid coordinates of the point cloud, with the reference point as the reference point.

[0048] One of the above technical solutions has the following advantages or beneficial effects:

[0049] This invention automates the entire inspection process by using a robotic arm to guide a structured light camera, eliminating the need for manual intervention and significantly improving inspection speed and efficiency. The robotic arm can move quickly and automatically scan different parts of the lifting device, avoiding blind spots in traditional methods and thus improving the comprehensiveness and coverage of the inspection. It employs non-contact 3D structured light imaging, so it does not directly contact the workpiece during inspection, avoiding surface damage that may be caused by contact measurements. Based on the characteristics of different inspection points, the appropriate use of RANSAC segmentation algorithms, Euclidean clustering algorithms, and edge detection can help identify the centroid of the inspection points on the lifting device, thereby improving the accuracy and efficiency of 3D inspection of the lifting device. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0051] Figure 1 This is a flowchart of the robotic arm-guided lifting device detection method provided in an embodiment of the present invention;

[0052] Figure 2 This is a flowchart of a robotic arm-guided lifting device detection method provided in another embodiment of the present invention;

[0053] Figure 3 These are detection effect diagrams at different points of the robotic arm-guided lifting device detection method provided in this embodiment of the invention;

[0054] Figure 4 This is a detection effect diagram of the robotic arm-guided lifting device detection system provided in an embodiment of the present invention;

[0055] Figure 5 This is an execution flowchart of the robotic arm-guided lifting device detection system provided in an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the structure of the robotic arm-guided lifting device detection system provided in an embodiment of the present invention. Detailed Implementation

[0057] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0058] In this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0059] In modern industrial production, the quality control and inspection of lifting equipment has become particularly important. Lifting equipment usually has irregular shapes and complex surface features. Traditional manual or traditional inspection methods (such as contact coordinate measuring machines, laser scanning, etc.) face many challenges under such complex conditions, such as: low inspection efficiency: traditional methods often require a lot of manual intervention, and for complex lifting equipment, it takes a long time to complete a comprehensive inspection; it is impossible to achieve multi-angle and all-round inspection, and for large-sized and complex-shaped lifting equipment, the limited field of view may result in some areas not being fully covered.

[0060] Common coordinate measuring machines (CMMs) acquire three-dimensional coordinates by having a probe contact the surface of a hanger. They are suitable for most simple workpieces, but they have many limitations for large, complex, and irregularly shaped workpieces: they are relatively slow, especially when fine measurements are required, and they are not flexible enough to perform all-round inspections of complex-shaped workpieces.

[0061] Laser scanning measurement, a traditional inspection method, is a non-contact measurement approach. It obtains three-dimensional information by projecting a laser beam onto the surface of a workpiece and collecting the position of the reflected laser. Laser scanning technology is used for measuring the shape of complex workpieces, but it is greatly affected by the surface material: laser scanning is less effective on smooth or highly reflective surfaces, and is prone to data loss or noise; it also has a limited field of view: fixed laser scanners can usually only scan from one perspective, making it difficult to achieve full coverage for complex-shaped workpieces, resulting in missing data in some areas; and it is the most expensive: laser scanning equipment and data processing are costly, especially in the inspection of large workpieces, which requires a large amount of equipment and data processing resources.

[0062] Therefore, a detection method based on robotic arm guidance is needed to detect lifting devices.

[0063] like Figure 1 As shown, in a preferred embodiment of the present invention, a method for detecting a lifting device guided by a robotic arm is provided. The robotic arm is equipped with a non-contact three-dimensional structured light camera. The robotic arm is moved to allow the camera to complete the detection according to a detection path. The method for detecting the lifting device includes the following steps:

[0064] S1: Select a reference point, a first detection point, a second detection point, and a third detection point in the lifting device according to the preset detection requirements;

[0065] S2: The camera acquires point clouds at the reference point, the first detection point, the second detection point, and the third detection point;

[0066] S3: Based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, calculate the centroid coordinates of the point cloud at the corresponding locations using at least two of the RANSAC segmentation algorithm, Euclidean clustering segmentation algorithm, and edge detection algorithm.

[0067] S4: Using the reference point as the reference point, obtain the error values ​​of the first detection point, the second detection point, and the third detection point based on the centroid coordinates of the point cloud.

[0068] Before step S1, OCR points can be detected first. The robotic arm moves to the OCR point location, and through cropping, filtering, and brightening, the entire lifting device is identified using the PaddleOCR model to obtain its number, facilitating identification and preventing confusion. In step S1, before inspecting the lifting device, multiple key points need to be selected according to preset inspection requirements for subsequent point cloud data acquisition and processing. These key points include a reference point and multiple detection points. The reference point serves as a reference for subsequent measurement, while the other detection points are used for precise measurement and error calculation of specific parts of the lifting device. The reference point is a fixed point used for reference, usually selected at a stable position on the lifting device, serving as the benchmark for all subsequent point measurements. The selection of the reference point is crucial, as it affects the accuracy of the entire inspection process. The first, second, and third detection points are points selected on the lifting device for specific measurements. Their positions and numbers depend on the design of the lifting device and the inspection requirements (e.g., different lifting devices have different purposes and functions, resulting in different selected detection points).

[0069] In step S2, during the robotic arm-guided detection, the robotic arm moves a structured light camera, which scans selected reference points and detection points to acquire their corresponding point cloud data. The point cloud data is a collection of all points on the object's surface; the three-dimensional coordinates of these points accurately describe the object's shape and structure. Point cloud data is a collection of points on the object's surface captured by a 3D sensor (such as a structured light camera). Each point has three coordinates (x, y, z), describing the spatial distribution of various locations on the object's surface. A structured light camera is a sensor that projects a known light structure (such as stripes or grids) onto the object's surface and captures the deformed image, thereby calculating the object's three-dimensional shape. Structured light cameras can provide high-precision point cloud data.

[0070] For the collected point cloud data, different algorithms are used for processing and analysis based on the characteristics of different points. RANSAC segmentation, Euclidean clustering, and edge detection algorithms are employed to separate noise points, fit planes, and identify object features from the point cloud data. After obtaining the correct point cloud region, the centroid coordinates of that region are calculated. RANSAC segmentation is an efficient and robust estimation algorithm that divides data into subsets of inliers and exliers. This algorithm designs a model based on the problem as the segmentation criterion, continuously inputs different data from the total dataset using random sampling, iteratively calculates the model parameters, and then re-divides the original dataset based on the model. Euclidean clustering divides the point cloud into different clusters based on the distance between points. Euclidean clustering is suitable for situations where the object surface is relatively uniformly distributed and can effectively remove background or irrelevant point clouds. After RANSAC algorithm segmentation removes a large amount of irrelevant environmental point cloud from the planes in a scene, it still cannot determine how many test objects exist in the scene or which cluster each point belongs to. Euclidean clustering essentially groups points that are close together into one class. Assuming there are n points in the point cloud, Euclidean distance is defined as the degree of closeness between two points, using the distance between nearest neighbors as the standard to achieve point cloud clustering and segmentation. Edge detection algorithms detect the edges of objects in the point cloud, helping to identify the object's contour and shape features. The centroid is the geometric center of an object or part of an object; centroid coordinates are used to describe the position of that part of the object and are usually obtained by calculating the average coordinates of the point cloud.

[0071] During the testing process, the centroid coordinates of different testing points are calculated, with the coordinates of the reference point serving as a reference. Error values ​​at other testing points are then calculated. Error value analysis allows for the assessment of the spreader's accuracy at various positions, thereby determining whether it meets design requirements. Figure 2 The following is a flowchart of the specific implementation of this embodiment, in which detection points 1 and 5 are the first detection points, detection points 2 and 4 are the second detection points, and detection point 3 is the third detection point.

[0072] This invention addresses the shortcomings of traditional three-coordinate contact measurement methods, such as low efficiency, limited flexibility, fragility, and the need for probes to contact the workpiece surface, which can easily damage the workpiece surface or the probe itself, especially when inspecting fragile or precision workpieces. Laser scanning measurement methods also suffer from drawbacks such as being greatly affected by surface materials, limited viewing angles, and high costs. To address these issues, a robotic arm-guided lifting fixture inspection method is proposed. By guiding a structured light camera with a robotic arm, the entire inspection process is automated, requiring no manual intervention and significantly improving inspection speed and efficiency. The robotic arm can move quickly and automatically scan different parts of the lifting fixture, avoiding blind spots in traditional methods and thus improving the comprehensiveness and coverage of the inspection. The use of non-contact three-dimensional structured light imaging avoids direct contact with the workpiece during inspection, preventing surface damage that can occur with contact measurements. Furthermore, by appropriately using RANSAC segmentation algorithms, Euclidean clustering algorithms, and edge detection based on the characteristics of different inspection points, the centroids of the inspection points on the lifting fixture can be identified, thereby improving the accuracy and efficiency of three-dimensional lifting fixture inspection.

[0073] Preferably, step S3 includes:

[0074] Based on the point cloud characteristics at the reference point, RANSAC segmentation is first performed on the point cloud at the reference point to obtain the first point cloud after removing the reference plane.

[0075] The first point cloud is segmented according to the Euclidean clustering segmentation algorithm, and the cluster point cloud with the largest center value Z value is selected as the second point cloud;

[0076] Perform RANSAC segmentation on the second point cloud to remove noise and identify the centroid, and obtain the centroid coordinates of the point cloud at the reference point;

[0077] The point cloud characteristics at the reference point include the connection between the reference point and its lower plane.

[0078] like Figure 3 As shown, the reference point is Figure 3The points in the first row of the data are connected to the lower plane (yellow area) and serve as reference points for calculating the coordinate deviations of other detection points. The Z-value represents the height of each point in the point cloud, corresponding to the Z-axis coordinate in 3D space. In this step, the clustered point cloud with the largest Z-value is selected as the second point cloud. Since the reference point is connected to the lower plane, if Euclidean clustering is directly applied to the reference point, undersegmentation will occur. Therefore, when processing the point cloud data, the RANSAC algorithm is first used to identify and remove the reference plane portion. By randomly selecting some points and fitting the plane, the RANSAC algorithm can effectively find data points consistent with the reference plane and remove these points, thereby removing ground interference (first point cloud) connected to the reference point and creating a clean point cloud data foundation for subsequent target object extraction. After the reference plane is removed, Euclidean clustering is used to segment the remaining point cloud. By setting a distance threshold, the algorithm groups points that are close to each other into one cluster. The cluster with the largest Z-value (height) is selected as the target point cloud (second point cloud), which effectively identifies the area where the target object is located, reduces the influence of background noise and irrelevant points, and ensures the accuracy of subsequent processing. For the selected target point cloud, the RANSAC algorithm is further used for fine segmentation to remove noise points, and the quality of the point cloud is optimized by fitting a more accurate model. Then, the centroid of the target point cloud is calculated to obtain its three-dimensional coordinates. This centroid position represents the position of the target object, and it is compared with the coordinates of the reference point to evaluate the error and ensure the accuracy of the positioning.

[0079] Preferably, step S3 includes:

[0080] Based on the point cloud characteristics at the first detection point, the Euclidean clustering segmentation algorithm processes the point cloud at the first detection point to obtain a clustered point cloud;

[0081] The centroid of each cluster point cloud is calculated. The RANSAC segmentation algorithm processes the cluster point cloud with the largest centroid Z value and then identifies the centroid coordinates at the first detection point.

[0082] Among them, the point cloud characteristics at the first detection point include the fact that the plane where the first detection point is located is connected to the impurity plane, resulting in the plane where the first detection point is located being uneven.

[0083] like Figure 3The second line shows the calculation process for the centroid of the first detection point. The first detection point is above the impurity plane, and the target plane above it is not very flat. The Euclidean clustering segmentation algorithm divides the point cloud data into multiple clusters by setting a distance threshold. Each cluster contains points with similar spatial locations. The goal of this process is to extract interconnected point groups from the complex point cloud, representing an object or a region. By calculating the distance between points, Euclidean clustering can effectively distinguish the points of the target object from background noise, thus laying the foundation for subsequent accurate segmentation. After completing Euclidean clustering, the next step is to calculate the centroid of each cluster. The centroid is obtained by calculating the mean of the coordinates of all points in the cluster, which represents the geometric center of the cluster. In this step, special attention is paid to the cluster with the largest Z-value (vertical coordinate), because the target object is usually located at a higher position and has a larger Z-value. In this way, the cluster most likely to represent the target object can be determined, thus preparing for the next step of RANSAC segmentation. After selecting the cluster with the largest Z-value, the RANSAC algorithm is applied to further refine the point cloud segmentation of this cluster. RANSAC randomly selects points and fits a planar model, removing points that do not conform to planar features and retaining point cloud data that truly belong to the target plane. Crucially, it accurately extracts the plane of the target object by removing noise and impurities. Especially when the target object's plane is connected to and uneven with impurity planes, RANSAC significantly improves segmentation accuracy, ensuring the target point cloud is accurately extracted. The final step in the target point cloud obtained through RANSAC segmentation is to calculate the centroid. The centroid is determined by calculating the mean coordinates of all points in the point cloud and is used to represent the object's position. The centroid calculation provides the target object with precise three-dimensional coordinates, helping to determine the target object's accurate location in space. It ensures that the point cloud data obtained through the preceding segmentation and extraction steps can be converted into specific spatial coordinates, thus supporting subsequent detection.

[0084] Preferably, step S3 includes:

[0085] Based on the point cloud characteristics at the second detection point, Euclidean clustering segmentation, RANSAC segmentation, edge detection, and Euclidean clustering segmentation are sequentially performed on the point cloud at the second detection point to obtain the outer ring point cloud and the inner ring point cloud.

[0086] Select the centroid coordinates of the point cloud at the second detection point by computing the inner ring point cloud;

[0087] Among them, the point cloud characteristics at the second detection point include the existence of two planes in the point cloud at the second detection point with an empty center.

[0088] like Figure 3As shown, the point cloud at the second detection point has two planes, and the center of the detection point is empty. Euclidean clustering coarse segmentation divides the point cloud into clusters based on the Euclidean distance between points. By setting a large distance threshold, the point cloud is initially grouped, with each cluster representing a potential object or region. The Euclidean clustering algorithm groups points that are close together into the same cluster, while points that are far apart are grouped into other clusters. The purpose of coarse segmentation is to initially separate different regions (outer and inner rings) in the point cloud from noisy data. At this stage, the segmentation accuracy requirement is not high; the goal is to extract the obviously different parts of the point cloud and eliminate some discrete noise points, laying the foundation for subsequent processing. Specifically, when performing Euclidean clustering, the algorithm calculates the similarity between each pair of points based on the distance between them and determines whether to group them into the same cluster based on the set threshold. In this way, the clustering algorithm divides the point cloud into multiple clusters, each representing a large region that may contain multiple objects or different parts of an object.

[0089] RANSAC plane segmentation randomly selects points and fits a plane model to filter out point clouds that conform to the plane assumption, thereby extracting planar structures from complex point cloud data. It can find groups of points that conform to the model even in noisy or complex data environments. RANSAC helps to eliminate points that do not conform to planar features and identifies the planar region around the second detection point, making it particularly suitable for extracting target planes in multiple planes or complex backgrounds. Since the point cloud at the second detection point will have two planes, and the center of the detection point is empty, edge detection is still required after segmentation. The purpose of edge detection is to identify areas with abrupt surface changes in the point cloud data, thereby extracting the edge information of the object. In point cloud processing, edge locations can be found by calculating the normals or curvatures of points. Areas with large changes in normals often represent the contours of objects. Edge detection helps to further extract the boundaries of target regions from the segmented planes, especially in complex scenes, clearly separating inner and outer rings and avoiding confusion and omissions.

[0090] Euclidean clustering for fine segmentation builds upon coarse segmentation by further refining the point cloud segmentation using a more precise clustering algorithm. By adjusting distance thresholds or other clustering parameters, fine segmentation can divide the target region at a finer granularity, ensuring clear separation between inner and outer rings. It is suitable for scenarios with high point cloud density or complex boundaries. By refining the segmentation results, it ultimately extracts accurate target regions, reducing the possibility of misclassification. Centroid calculation is performed by averaging the coordinates of all point clouds within the target region to obtain the geometric center of that region. By calculating the centroid of the inner ring point cloud, the 3D coordinates of the second detection point can be obtained, serving as a representative position of the target region. The centroid, as the basis for object localization, provides an accurate reference point in subsequent processing or analysis, providing crucial data for subsequent target recognition or localization tasks and ensuring the system can correctly identify and process the target region.

[0091] Preferably, the point cloud characteristics at the third detection point include the existence of a plane behind the third detection point;

[0092] Step S3 includes:

[0093] Based on the point cloud characteristics at the third detection point, Euclidean clustering and RANSAC segmentation are performed on the point cloud at the third detection point, and then the centroid coordinates of the point cloud at the third detection point are calculated.

[0094] The equation of the back plane at the third detection point is obtained by using the RANSAC fitting plane model method of the original point cloud before performing Euclidean clustering segmentation and RANSAC segmentation.

[0095] Based on the equation and the centroid coordinates of the point cloud at the third detection point, the distance from the centroid of the point cloud at the third detection point to the plane behind the third detection point is calculated.

[0096] The RANSAC algorithm is used to fit a plane. A subset of points in the point cloud is randomly selected to fit a planar model, and the degree of matching between the model and the data is evaluated. Finally, the equation of the rear plane is extracted. Centroid calculation is achieved by finding the geometric center of the target point cloud region, i.e., the average coordinates of all points in that region. For the third detection point, the centroid is first calculated within the precise point cloud region obtained after Euclidean clustering and RANSAC segmentation. The RANSAC method is then used to fit a plane to the original point cloud data, thus obtaining the mathematical equation of the rear plane. The RANSAC algorithm continuously optimizes the planar model by randomly selecting points from the data and calculating the planar model, ultimately obtaining a planar equation that best fits most point cloud data. The distance from the centroid to the rear plane is calculated using the distance formula between the plane equation and the point, based on the fitted planar equation and the calculated centroid coordinates. By solving for the perpendicular distance between the centroid coordinates and the plane equation, the shortest distance between the centroid and the plane can be obtained. This step is used to determine the relative positional change between the target point cloud and the rear plane, and to monitor whether the point cloud has been displaced. This distance value can be used as a measure of displacement to help evaluate whether the changes in the point cloud in the system meet expectations, which is beneficial for detection.

[0097] Preferably, when the RANSAC segmentation algorithm segments a plane, the points on the plane to be segmented satisfy the following relationship:

[0098] ax + by + cz = d;

[0099] Where (x,y,z) represents the three-dimensional coordinates of any point in the plane, and (a,b,c) represents the direction of the plane, satisfying (a 2 +b 2 +c 2 ) = 1, where d represents the distance from the point cloud location to the plane;

[0100] When converting planar parameters into a matrix, let the expression for the parameters of the plane to be measured be: ∏=[a,b,c,d] T Three points are randomly extracted from the point cloud data using a random sampling method, satisfying the following relationship:

[0101]

[0102] Where P represents any point in the scene, ∏ represents a plane, and (x, y, z) represents the three-dimensional coordinates of any point on the plane. Describe the calculated parameters and satisfy:

[0103] Set a distance threshold δ0. If the distance from any point P in the scene to the plane ∏ satisfies d < δ0, the point is recorded as an in-place point; otherwise, it is recorded as an out-of-place point.

[0104] Specifically, RANSAC randomly selects the smallest dataset to calculate the model parameters, then evaluates the fit of all data points to the model, calculates the number of inliers and outliers, and through multiple iterations, selects the model with the most inliers as the final result, thereby effectively eliminating outliers and finding the best-fit model.

[0105] Preferably, the Euclidean clustering segmentation algorithm includes:

[0106] For a point cloud dataset P, determine a query point P. i Set a distance threshold r, where the formula for calculating the threshold r is:

[0107]

[0108] Where, p i With p j Let p represent two points in a point cloud. ik With p jk This represents any point in the neighborhood of two points;

[0109] Find the n nearest neighbors P of the KD-Tree. j (j=1,2,...,n), calculate the Euclidean distance d from the n nearest neighbors to the query point according to the formula for calculating the threshold r. j Compare distance d j Points with a distance threshold r are assigned to class M, and the segmentation is completed when the number of points in class M no longer increases.

[0110] Specifically, the algorithm first selects a point as the query point and sets a distance threshold. Only other points that are close to the query point will be classified into the same category. By calculating the Euclidean distance between the query point and other points, it can be determined which points belong to the same cluster. To improve efficiency, the algorithm uses data structures such as KD-Tree to accelerate the search for nearest neighbors and avoids calculating all points one by one. Then, the algorithm continuously expands the current cluster, gradually adding points in the neighborhood that meet the conditions to the current cluster until the cluster can no longer be expanded. Finally, all points will be classified into different clusters, completing the segmentation of the point cloud.

[0111] Furthermore, the point clouds at the reference point, the first detection point, the second detection point, and the third detection point obtained in step S2 are all ROI point clouds obtained by using a pass-through filter.

[0112] The point clouds at the reference point, first detection point, second detection point, and third detection point are all ROI (Region of Interest) point clouds extracted from the original point cloud data using the pass-through filter method. Pass-through filtering is a commonly used point cloud data processing technique that filters point cloud data located within a specified region by setting minimum and maximum filtering ranges along specific coordinate axes. In this way, only point clouds within the ROI are retained, while irrelevant background or noise is removed, thereby improving the accuracy and efficiency of subsequent processing. The reference point and detection point represent key locations or feature points within that region, used for further detection.

[0113] like Figure 6 A robotic arm-guided lifting device detection system, applied to the lifting device detection method described above, characterized in that the lifting device detection system comprises:

[0114] The point selection module is used to select a reference point, a first detection point, a second detection point, and a third detection point in the lifting device according to preset detection requirements;

[0115] The point cloud acquisition module is used to acquire point clouds at the reference point, the first detection point, the second detection point, and the third detection point;

[0116] The coordinate calculation module is used to calculate the centroid coordinates of the point cloud at the corresponding location based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, using at least two of the RANSAC segmentation algorithm, the Euclidean clustering segmentation algorithm, and the edge detection algorithm.

[0117] The error detection module is used to obtain the error values ​​of the first detection point, the second detection point, and the third detection point based on the centroid coordinates of the point cloud, with the reference point as the reference point.

[0118] Figure 4 This is a diagram illustrating the inspection results of a robotic arm-guided lifting device inspection system. Figure 5 This is a flowchart of one embodiment of a robotic arm-guided lifting device detection system. This embodiment implements the lifting device detection method and process. Please refer to the above embodiments for details, which will not be repeated here.

[0119] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0120] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for detecting a lifting device guided by a robotic arm, characterized in that, The robotic arm is equipped with a three-dimensional structured light camera. The robotic arm is used to move so that the camera completes the detection according to the detection path. The lifting device detection method includes the following steps: S1: Select a reference point, a first detection point, a second detection point, and a third detection point in the lifting device according to the preset detection requirements; S2: The camera acquires point clouds at the reference point, the first detection point, the second detection point, and the third detection point; S3: Based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, calculate the centroid coordinates of the point cloud at the corresponding locations using at least two of the RANSAC segmentation algorithm, Euclidean clustering segmentation algorithm, and edge detection algorithm. S4: Using the reference point as the reference point, obtain the error values ​​of the first detection point, the second detection point, and the third detection point based on the centroid coordinates of the point cloud; Step S3 includes: Based on the point cloud characteristics at the reference point, RANSAC segmentation is first performed on the point cloud at the reference point to obtain the first point cloud after removing the reference plane. The first point cloud is segmented according to the Euclidean clustering segmentation algorithm, and the cluster point cloud with the largest center value Z value is selected as the second point cloud; Perform RANSAC segmentation on the second point cloud to remove noise and identify the centroid, and obtain the centroid coordinates of the point cloud at the reference point; Among them, the point cloud characteristics at the reference point include the connection between the reference point and its lower plane; Step S3 includes: Based on the point cloud characteristics at the first detection point, the Euclidean clustering segmentation algorithm processes the point cloud at the first detection point to obtain a clustered point cloud; The centroid of each cluster point cloud is calculated. The RANSAC segmentation algorithm processes the cluster point cloud with the largest centroid Z value and then identifies the centroid coordinates at the first detection point. Among them, the point cloud characteristics at the first detection point include the fact that the plane where the first detection point is located is connected to the impurity plane, resulting in the plane where the first detection point is located being uneven. Step S3 includes: Based on the point cloud characteristics at the second detection point, Euclidean clustering segmentation, RANSAC segmentation, edge detection, and Euclidean clustering segmentation are sequentially performed on the point cloud at the second detection point to obtain the outer ring point cloud and the inner ring point cloud. Select the centroid coordinates of the point cloud at the second detection point by computing the inner ring point cloud; Among them, the point cloud characteristics at the second detection point include that the point cloud at the second detection point has two planes and the center is empty; The point cloud characteristics at the third detection point include the existence of a plane behind the third detection point; Step S3 includes: Based on the point cloud characteristics at the third detection point, Euclidean clustering and RANSAC segmentation are performed on the point cloud at the third detection point, and then the centroid coordinates of the point cloud at the third detection point are calculated. The equation of the back plane at the third detection point is obtained by using the RANSAC fitting plane model method of the original point cloud before performing Euclidean clustering segmentation and RANSAC segmentation. Based on the equation and the centroid coordinates of the point cloud at the third detection point, the distance from the centroid of the point cloud at the third detection point to the plane behind the third detection point is calculated.

2. The lifting gear testing method according to claim 1, characterized in that, RANSAC partitioning algorithm When dividing a plane, points on the plane to be divided satisfy the following relationship: in, Represents the three-dimensional coordinates of any point in a plane. Represents the direction of a plane, and satisfies , This indicates the distance from the point in the point cloud to the plane. When converting plane parameters into matrices, let the expression for the parameters of the plane to be measured be: Three points are randomly extracted from the point cloud data using a random sampling method, satisfying the following relationship: ； in, Represents any point within the scene. Represents a plane. Represents the three-dimensional coordinates of any point in a plane. Describe the calculated parameters and satisfy: ; Set distance threshold If any point in the scene to plane The distance satisfies If the condition is met, the point is recorded as an in-place point; otherwise, it is recorded as an out-of-place point.

3. The lifting gear testing method according to claim 1, characterized in that, Euclidean clustering segmentation algorithms include: For point cloud datasets Determine a query point Set distance threshold , where the threshold The calculation formula is: ； in, and This represents two points in a point cloud. and This represents any point in the neighborhood of two points; Find the nearest one using the KD-Tree. Neighboring points According to the threshold formula The calculation formula is used to calculate Euclidean distance from the nearest neighbor to the query point Compare distances With distance threshold , will be less than The points are classified into categories. In, all the way to class The segmentation is complete when the number of points in the segment no longer increases.

4. The method for testing lifting devices according to any one of claims 1-3, characterized in that, The point clouds at the reference point, the first detection point, the second detection point, and the third detection point obtained in step S2 are all ROI point clouds obtained by using a pass-through filter.

5. A robotic arm-guided lifting device inspection system, applied to the lifting device inspection method according to any one of claims 1-4, characterized in that, The lifting device detection system includes: The point selection module is used to select a reference point, a first detection point, a second detection point, and a third detection point in the lifting device according to preset detection requirements; The point cloud acquisition module is used to acquire point clouds at the reference point, the first detection point, the second detection point, and the third detection point; The coordinate calculation module is used to calculate the centroid coordinates of the point cloud at the corresponding location based on the point cloud characteristics at the reference point, the first detection point, the second detection point, and the third detection point, using at least two of the RANSAC segmentation algorithm, the Euclidean clustering segmentation algorithm, and the edge detection algorithm. The error detection module is used to obtain the error values ​​of the first detection point, the second detection point, and the third detection point based on the centroid coordinates of the point cloud, with the reference point as the reference point.

Images