Quality detection method, equipment and storage medium

Abstract

The invention discloses a quality detection method, a medium and a readable storage medium. The method comprises the steps that firstly, a transformation matrix from a preset sensor coordinate system to a robot tail end coordinate system is obtained; based on the transformation matrix, uniformly converting three-dimensional point cloud data obtained by scanning a to-be-measured workpiece to a robot-based coordinate system; processing the point cloud data under the base coordinate system to extract geometric parameters of the workpiece target features; and judging the quality of the workpiece according to a comparison result of the geometric parameters and a preset standard. According to the invention, through high-precision coordinate unification and automatic feature extraction, the accuracy and efficiency of detection are improved.

Description

Technical Field

[0001] This application relates to the fields of industrial automation and machine vision technology in industrial manufacturing, specifically to a quality inspection method, equipment, and storage medium. Background Technology

[0002] Non-contact quality inspection technology based on robots and 3D vision sensors has become an important means of realizing online inspection in industrial automation. This technology uses a robot equipped with a 3D sensor to acquire workpiece point cloud data, automatically extracting geometric features and completing quality judgment.

[0003] However, existing technologies have several prominent shortcomings in practical applications, which limit the detection accuracy and system reliability: In hand-eye calibration, existing methods suffer from insufficient accuracy and robustness. Traditional calibration methods rely on high-precision fixtures and manual operation, which are cumbersome and prone to introducing errors. Automatic calibration algorithms based on multiple sets of point cloud data struggle to balance noise robustness and the orthogonality constraints of the rotation matrix when solving for the transformation matrix, easily getting trapped in local optima. This results in limited accuracy of the calibration results and becomes a major source of systematic error.

[0004] In point cloud data processing and feature extraction, the complex environment of industrial sites poses challenges to existing technologies. Raw point clouds often contain noise and outliers, and the data is incomplete due to workpiece structure or viewpoint occlusion. Current methods lack robustness when processing key features such as holes and edges: for point clouds with hole features that have burrs, oil stains, or partial occlusion, the fitted center coordinates and hole diameter dimensions are unstable; when calculating edge spacing, the fitting results are easily affected by abnormal jumps in the point cloud, and measurement repeatability is difficult to guarantee.

[0005] In terms of system intelligence and adaptability, existing solutions generally lack flexibility. Detection paths and algorithm parameters often need to be pre-set, resulting in poor adaptability to changes in workpiece pose and environmental disturbances. When product models or detection features change, complex recalibration and parameter adjustments by professionals are required, leading to high system deployment and maintenance costs.

[0006] Therefore, how to overcome the shortcomings of insufficient hand-eye calibration accuracy, poor robustness of complex point cloud feature extraction, and weak system adaptability, and provide a high-precision, highly reliable, and easy-to-deploy robot quality inspection method has become an urgent technical problem to be solved in this field. Summary of the Invention

[0007] The purpose of this application is to provide a quality inspection method, device, and storage medium. By employing a high-precision coordinate system technology, it addresses the problems of low detection accuracy and poor robustness in existing robot-based visual inspection methods caused by hand-eye calibration errors and feature extraction interference.

[0008] To address the aforementioned technical problems, the first aspect of this application discloses a quality inspection method, comprising: Obtain the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system; Based on the transformation matrix, the three-dimensional point cloud data obtained from scanning the workpiece under test is uniformly transformed to the base coordinate system; In the base coordinate system, the point cloud data is processed to extract the geometric parameters of the target features of the workpiece; The quality of the workpiece is determined based on the comparison results between the geometric parameters of the target feature and the preset standard.

[0009] Optionally, obtaining the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system includes the following steps: The robot is controlled to carry sensors to scan a fixed calibration object in multiple different postures, and to collect multiple sets of data simultaneously. Each set of data includes a frame of three-dimensional point cloud and the robot pose of the robot end in the base coordinate system when the three-dimensional point cloud is collected. Extract the feature point coordinates of the calibrated object in the preset sensor coordinate system from each frame point cloud; For each set of data, a set of theoretical coordinates for the calibration object is determined based on the coordinates of the feature points and the corresponding robot pose. Based on multiple sets of theoretical coordinates, construct an optimization problem for the transformation matrix; The transformation matrix is ​​obtained by solving the optimization problem using an optimization algorithm.

[0010] Optionally, solving the optimization problem using an optimization algorithm to obtain the transformation matrix includes the following steps: Candidate parameters for the transformation matrix are generated using a particle swarm optimization algorithm. In each iteration of the particle swarm optimization algorithm, the augmented Lagrange method is used to locally optimize the current candidate parameters; The particle state is updated based on the results of local optimization until the particle state satisfies the convergence condition, and then the transformation matrix is ​​output.

[0011] Optionally, the step of using the augmented Lagrange method to locally optimize the current candidate parameters includes: An augmented Lagrangian function is constructed based on minimizing the deviation between the theoretical and actual coordinates of the feature points of the calibration object under different poses and the orthogonality of the rotation matrix in the transformation matrix. By solving the augmented Lagrangian function, local optimization of the candidate parameters can be achieved.

[0012] Optionally, if the fitness value of the current generation's best particle is not better than the global best value, then calculate the fitness value of the current generation's worst particle. If the worst particle fitness value is better than the current global optimum, then the particle state is determined to satisfy the convergence condition.

[0013] Optionally, the target feature includes a hole; When the target feature is a hole, the steps for extracting the geometric parameters include: Multi-level planar fitting and projection filtering are performed on the point cloud data to obtain a denoised hole point cloud. Curvature calculation and spatial continuity analysis are performed on the denoised hole point cloud to screen out candidate points at the hole edges; The selected candidate points at the edge of the hole are fitted with an ellipse or a circle, and the fitted center is used as the coordinate of the hole center.

[0014] Optionally, performing multi-level planar fitting and projection filtering on the point cloud data to obtain the denoised hole point cloud includes the following steps: Based on a first distance threshold, a first reference plane is obtained by performing plane fitting on the point cloud data. Discrete points whose distance to the first reference plane is greater than the first distance threshold are removed; The points retained after removal are projected onto a second reference plane fitted based on a second distance threshold to obtain the denoised hole point cloud; Wherein, the first distance threshold is greater than the second distance threshold.

[0015] Optionally, the step of performing curvature calculation and spatial continuity analysis on the denoised hole point cloud to screen out candidate points at the hole edges includes the following steps: Calculate the curvature value of each point in the denoised point cloud; Analyze the continuity of depth between adjacent points in a point cloud sequence, and identify and eliminate depth jump points caused by occlusion; Based on the curvature threshold, points with larger curvature are selected from non-depth transition points as candidate points for the hole edge.

[0016] Optionally, the target feature includes edges, and the geometric parameter is the edge spacing; The steps for extracting the edge spacing include: The point sets of the two sides are separated from the point cloud data; Fit a reference line based on the set of points on one side of the edge; Calculate the distance from each point in the edge point set on the other side to the reference line; Outlier removal is performed on the calculated distance set based on the 3σ principle of normal distribution; The average distance after removing the effective distances is calculated as the edge spacing.

[0017] Secondly, embodiments of this application provide an apparatus comprising a processor and a memory storing a computer program, wherein the steps of the quality detection method are implemented when the processor runs the computer program.

[0018] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the quality detection method.

[0019] This application provides a quality inspection method, equipment, and storage medium. By combining high-precision hand-eye calibration with robust feature extraction, it achieves automated and precise inspection of workpiece geometric parameters. The scheme first obtains a high-precision hand-eye transformation matrix through an optimized algorithm, establishing a precise mapping relationship between the sensor and robot coordinate systems. Then, using this matrix, the workpiece's 3D point cloud is uniformly transformed to a base coordinate system. A feature extraction algorithm then accurately extracts key geometric parameters such as hole center coordinates and edge spacing. Finally, quality judgment is achieved through parameter comparison. This method effectively solves the problem of insufficient accuracy caused by large calibration errors and weak anti-interference ability of feature extraction in traditional inspection, significantly improving the reliability and stability of the inspection results. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0021] Figure 1 This is a flowchart illustrating the quality testing method provided in an embodiment of this application.

[0022] Figure 2 This is a schematic diagram of the process for obtaining the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system as described in the embodiments of this application.

[0023] Figure 3 This is a schematic diagram illustrating the process of solving the optimization problem using an optimization algorithm to obtain the transformation matrix, as described in the embodiments of this application.

[0024] Figure 4 This is a schematic diagram of the process for extracting the geometric parameters when the target feature is a hole, as described in an embodiment of this application.

[0025] Figure 5 This is a schematic diagram illustrating the process of performing multi-level planar fitting and projection filtering on the point cloud data to obtain a denoised hole point cloud, as described in an embodiment of this application.

[0026] Figure 6 This is a schematic diagram illustrating the process of performing curvature calculation and spatial continuity analysis on the denoised hole point cloud to screen out candidate points at the hole edges, as described in an embodiment of this application.

[0027] Figure 7 This is a schematic diagram of the process for extracting edge spacing as described in an embodiment of this application.

[0028] Figure 8 This is a block diagram of the device described in the embodiments of this application.

[0029] The realization of the objectives, functional features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and textual descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0030] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0031] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, components, features, and elements with the same names in different embodiments of this application may have the same meaning or different meanings, the specific meaning of which must be determined by its interpretation in that specific embodiment or further in conjunction with the context of that specific embodiment.

[0032] It should be understood that although the terms first, second, third, etc., may be used herein to describe various information, such information should not be limited to these terms. These terms are used only to distinguish information of the same type from one another. For example, without departing from the scope of this document, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if," as used herein, can be interpreted as "when," "when," or "in response to determination." Furthermore, as used herein, the singular forms "a," "an," and "the" are intended to also include the plural forms unless the context indicates otherwise. It should be further understood that the terms "comprising," "including," indicate the presence of the stated feature, step, operation, element, component, item, kind, and / or group, but do not exclude the presence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, kinds, and / or groups. The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination thereof. Therefore, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C". Exceptions to this definition will only occur if the combination of elements, functions, steps, or operations is inherently mutually exclusive in some way.

[0033] It should be understood that although the steps in the flowcharts of this application's embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0034] It should be noted that step designations such as S11 and S12 are used in this document for the purpose of more clearly and concisely describing the corresponding content, and do not constitute a substantial limitation on the order. In specific implementation, those skilled in the art may execute S12 first and then S11, etc., but these should all be within the protection scope of this application.

[0035] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0036] In the following description, the use of suffixes such as "module," "part," or "unit" to denote elements is solely for the purpose of illustrative purposes and has no specific meaning in itself. Therefore, "module," "part," or "unit" may be used interchangeably.

[0037] This application discloses a quality inspection method applied to the inspection of the sliding rails of the driver's cab's front sliding window. Specifically, the core of the welding quality inspection of the sliding rails of the driver's cab lies in the precise measurement of the center-to-center distance of the positioning mounting holes on the rails and the spacing between the parallel tracks. Due to the large size of the rail structure (its length can reach approximately 900 mm) and the complex spatial distribution of the features to be inspected, it is usually necessary to use a robot carrying vision sensors to enter the confined space inside the driver's cab for automated measurement.

[0038] Currently, the technical approach to achieving this automated measurement faces a dual challenge: if a structured light camera capable of single-shot imaging is used, its measurement accuracy is low, making it difficult to meet the requirements of high-precision detection; if a 3D laser contour sensor with higher measurement accuracy is used, a robot is required to move and scan the sensor, and the final measurement accuracy is highly dependent on the accuracy of hand-eye calibration (i.e., determining the transformation relationship between the sensor coordinate system and the robot's end-effector coordinate system). However, traditional hand-eye calibration methods are limited by their solution algorithms, easily getting trapped in local optima, and thus failing to achieve the high accuracy required for this application scenario.

[0039] Furthermore, the inherent characteristics of the workpiece under test further increase the difficulty of detection. For example, the threads inside the positioning mounting holes will generate significant noise in the point cloud data, and there are also machining defects or point cloud noise at the edge of the track. These factors seriously interfere with the accurate extraction of feature points (such as the center of the hole and the edge), leading to increased measurement errors.

[0040] Therefore, addressing the aforementioned issues, this application tackles the technical challenges in automated quality inspection of the driver's cab sliding window rail. It improves the calibration accuracy of the 3D laser contour sensor through an improved hand-eye calibration method and employs advanced optimization algorithms to ensure the accuracy and stability of the calibration process. Secondly, by introducing multi-level filtering and intelligent feature extraction technology, it effectively overcomes the interference of complex noise environments on feature extraction, significantly improving the accuracy of hole center positioning and edge recognition. The implementation of this solution will greatly improve the accuracy and reliability of sliding window rail welding quality inspection, providing effective technical support for quality control in the automotive manufacturing industry.

[0041] See Figure 1 , Figure 1 This is a flowchart illustrating the quality inspection method provided in this embodiment. The following are the specific implementation steps of this embodiment, which include the following steps: S101: Obtain the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system.

[0042] The transformation matrix is ​​used to transform the point cloud data collected by the sensor from the sensor coordinate system to the robot end effector coordinate system.

[0043] The sensor coordinate system refers to a coordinate system established with the optical center of the 3D vision sensor as the origin, and the direction of the coordinate axes of the sensor coordinate system is determined by the internal parameters of the sensor. The robot end effector coordinate system refers to a coordinate system established with the center of the flange of the robot end effector as the origin, and the direction of the coordinate axes of the robot end effector coordinate system maintains a fixed relationship with the robot base coordinate system. By using the transformation matrix between the two coordinate systems obtained through calibration, point cloud data can be transformed from the sensor coordinate system to the robot end effector coordinate system, laying the foundation for subsequent transformation to the base coordinate system.

[0044] For example, the transformation matrix is ​​obtained by controlling a robot carrying sensors to scan a fixed calibration object in multiple different postures, simultaneously collecting multiple sets of data. Each set of data includes a frame of 3D point cloud and the pose of the robot's end effector in the base coordinate system when the frame of 3D point cloud is collected; extracting the feature point coordinates of the calibration object in the sensor coordinate system from each frame of point cloud; for each set of data, determining a set of theoretical coordinates of the calibration object based on the feature point coordinates and the corresponding robot pose; constructing an optimization problem for the transformation matrix based on multiple sets of theoretical coordinates; and solving the optimization problem using an optimization algorithm to obtain the transformation matrix.

[0045] S102: Based on the transformation matrix, the three-dimensional point cloud data obtained from scanning the workpiece to be tested is uniformly transformed to the base coordinate system.

[0046] The transformation includes sequentially transforming the point cloud coordinates in the sensor coordinate system through the transformation matrix from the sensor to the robot end effector and the transformation matrix from the robot end effector to the base coordinate system to obtain the point cloud coordinates in the base coordinate system.

[0047] For example, for each point cloud point P_sensor acquired by the sensor, its coordinate P_base in the base coordinate system is calculated by the formula P_base = T_robot_end * T_end_sensor * P_sensor, where T_robot_end is the pose transformation matrix of the robot end effector in the base coordinate system, and T_end_sensor is the transformation matrix obtained in step S101.

[0048] S103: In the base coordinate system, process the point cloud data to extract the geometric parameters of the target features of the workpiece.

[0049] The target features include at least one of holes, edges, and planes.

[0050] Specifically, when the target feature is a hole, the steps for extracting the geometric parameters include: performing multi-level plane fitting and projection filtering on the point cloud data to obtain a denoised hole point cloud; performing curvature calculation and spatial continuity analysis on the denoised hole point cloud to screen out candidate points for hole edges; and performing ellipse or circle fitting on the screened candidate points for hole edges, using the fitting center as the coordinates of the hole center.

[0051] For example, the multi-level plane fitting and projection filtering includes: performing plane fitting on the point cloud data based on a first distance threshold to obtain a first reference plane; removing discrete points whose distance to the first reference plane is greater than the first distance threshold; and projecting the points retained after removal onto a second reference plane obtained based on a second distance threshold to obtain the denoised hole point cloud; wherein the first distance threshold is greater than the second distance threshold.

[0052] S104: Determine the quality of the workpiece based on the comparison results between the geometric parameters of the target feature and the preset standard.

[0053] The target features refer to key structures on the workpiece with specific geometric shapes used for quality evaluation, including but not limited to features such as positioning holes, track edges, and mounting planes. The comparison includes calculating the deviation between the extracted geometric parameters and preset standard values, and determining whether the deviation is within the allowable tolerance range. When the geometric parameter deviations of all target features are within the corresponding tolerance range, the workpiece is deemed to be of acceptable quality; if the geometric parameter deviation of any target feature exceeds the tolerance range, the workpiece is deemed to be of unacceptable quality.

[0054] For example, when the target feature is a hole, the hole center coordinates and hole diameter are compared with a preset standard; when the target feature is an edge, the edge spacing is compared with a preset standard; if all parameter deviations are within the allowable tolerance range, the workpiece is deemed qualified; if any parameter exceeds the tolerance range, the workpiece is deemed unqualified.

[0055] Through the above steps, this embodiment realizes robot-based automated quality inspection. Through precise hand-eye calibration and robust point cloud processing, it can accurately extract the geometric parameters of key features of the workpiece and automatically determine the quality of the workpiece according to preset standards, thereby improving inspection efficiency and accuracy.

[0056] See Figure 2 , Figure 2 This is a schematic diagram of the process for obtaining the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system as described in the embodiments of this application.

[0057] Step S101: Obtaining the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system includes the following steps: S201: Control the robot to carry sensors to scan the fixed calibration object in multiple different postures, and simultaneously collect multiple sets of data. Each set of data includes a frame of three-dimensional point cloud and the robot pose of the robot end in the base coordinate system when the three-dimensional point cloud is collected.

[0058] The multiple different postures include at least multiple different robot end-effector poses to ensure that the sensor’s viewpoint and distance relative to the calibration object vary sufficiently to improve calibration accuracy.

[0059] S202: Extract the feature point coordinates of the calibration object in the preset sensor coordinate system from the point cloud of each frame.

[0060] The feature point coordinates include the three-dimensional coordinates of known geometric features on the calibration object in the sensor coordinate system.

[0061] For example, when the calibration object is a sphere, the coordinates of the sphere's center are obtained by fitting the point cloud using the least squares method; when the calibration object is a planar calibration plate, the three-dimensional coordinates of preset marker points on the calibration plate are extracted.

[0062] S203: For each set of data, determine a set of theoretical coordinates for the calibration object based on the coordinates of the feature points and the corresponding robot pose.

[0063] The process of determining the theoretical coordinates is a crucial step in constructing the objective function to solve for the transformation matrix. The method involves mapping the coordinates of feature points in the sensor coordinate system to the base coordinate system using an initial or candidate transformation matrix to be optimized, combined with the known robot end-effector pose at that posture, thereby obtaining a set of calculated theoretical coordinates.

[0064] For example, for the i-th set of data, the formula for calculating its theoretical coordinates P_base_i(cal) is: P_base_i(cal) = T_base_end_i * T_end_sensor(cand) * P_sensor_i. Where P_sensor_i is the coordinates of the feature point extracted in the sensor coordinate system, T_base_end_i is the known transformation matrix from the robot end effector to the base coordinate system in the i-th pose, and T_end_sensor(cand) is the candidate transformation matrix currently being iteratively optimized. This formula can generate a set of corresponding theoretical coordinates for each candidate transformation matrix in the optimization process. S204: Based on multiple sets of theoretical coordinates, construct an optimization problem for the transformation matrix.

[0065] The objective function of the optimization problem is to minimize the sum of squared errors between multiple sets of theoretical coordinates calculated from all candidate transformation matrices and the known actual coordinates of the calibration object feature points in the base coordinate system. The constraints include the orthogonality of the rotation matrices in the transformation matrix.

[0066] For example, the solution process of the optimization problem includes: calculating the sum of squared errors between theoretical coordinates and actual coordinates in all sets of data, minimizing the sum of errors by adjusting the parameters of the transformation matrix, while ensuring the orthogonality of the rotation matrix.

[0067] S205: Solve the optimization problem using an optimization algorithm to obtain the transformation matrix.

[0068] The optimization algorithm can be a hybrid optimization strategy that combines particle swarm optimization and augmented Lagrange multiplication.

[0069] For example, the solution process includes: generating candidate parameters for the transformation matrix using a particle swarm optimization algorithm; in each iteration, using the augmented Lagrange method to locally optimize the current candidate parameters and handle the orthogonality constraints of the rotation matrix; iterating until the convergence condition is met, and outputting the final, optimal transformation matrix.

[0070] This embodiment achieves high-precision and robust calibration of the sensor to the robot end-effector coordinate system through multi-pose data acquisition, feature point extraction, theoretical coordinate calculation, optimization problem construction, and hybrid optimization solution, laying the foundation for accurate conversion of subsequent workpiece point cloud data.

[0071] See Figure 3 , Figure 3 This is a schematic diagram illustrating the process of solving the optimization problem using an optimization algorithm to obtain the transformation matrix, as described in the embodiments of this application.

[0072] Step S205: Solve the optimization problem using an optimization algorithm to obtain the transformation matrix. This includes the following steps: S301: The candidate parameters of the transformation matrix are generated using the particle swarm optimization algorithm.

[0073] The candidate parameters include the translation vector and rotation vector in the transformation matrix.

[0074] Specifically, the particle swarm is initialized, with each particle representing a solution to a candidate transformation matrix, including 6-dimensional parameters (3-dimensional translation vector and 3-dimensional rotation vector).

[0075] For example, the particle swarm size is set to 50, the initial position of each particle is randomly generated in the parameter space, and the initial velocity is zero; the maximum number of iterations is set to 200, the inertia weight is 0.9, and the learning factors c1 and c2 are both 2.0.

[0076] S302: In each iteration of the particle swarm optimization algorithm, the augmented Lagrange method is used to locally optimize the current candidate parameters.

[0077] The purpose of the local optimization is to further reduce the objective function value while maintaining the orthogonality of the rotation matrix.

[0078] Specifically, for each particle in the current iteration, an augmented Lagrangian function is constructed, and the orthogonality constraint of the rotation matrix is ​​transformed into a penalty term and added to the objective function; a local search is performed on the current candidate parameters using gradient descent or Newton's method to obtain a better local solution.

[0079] For example, the augmented Lagrangian function comprises two parts: the first part is the sum of squared errors between the theoretical coordinates and the actual coordinates; the second part is the penalty term for the orthogonality constraint of the rotation matrix, in the form λ||R^TR - I||^2 + μ||det(R) - 1||^2, where λ and μ are penalty coefficients.

[0080] S303: Update the particle state based on the results of local optimization until the particle state satisfies the convergence condition, and output the transformation matrix.

[0081] The particle state update includes updating the particle's position, velocity, and fitness value.

[0082] Specifically, based on the update rules of the particle swarm optimization algorithm and combined with the solution after local optimization, the position and velocity of each particle are updated; the fitness value (i.e. the objective function value) of the updated particle is calculated; and the current best particle and the global best particle are recorded.

[0083] For example, the convergence conditions include: the fitness value of the globally optimal particle changes less than a preset threshold (e.g., 1e-6) for 10 consecutive generations, or the maximum number of iterations is reached. When the convergence conditions are met, the transformation matrix corresponding to the globally optimal particle is output as the final solution.

[0084] Through the above embodiments, this application achieves high-precision hand-eye calibration based on a hybrid optimization strategy. By leveraging the global search capability of the particle swarm optimization algorithm and the local optimization capability of the augmented Lagrange method, it effectively avoids getting trapped in local optima while ensuring the orthogonality constraint of the rotation part in the transformation matrix, thereby improving the accuracy and robustness of the calibration.

[0085] Furthermore, step S302: the local optimization of the current candidate parameters using the augmented Lagrangian method includes: constructing an augmented Lagrangian function based on minimizing the deviation between the theoretical and actual coordinates of the calibration object feature points under different poses and the orthogonality of the rotation matrix in the transformation matrix; and achieving local optimization of the candidate parameters by solving the augmented Lagrangian function.

[0086] Specifically, the construction process of the augmented Lagrangian function includes: using the sum of squared deviations between theoretical and actual coordinates as the main objective function term; and converting the orthogonality constraint R^TR = I and the determinant constraint det(R) = 1 of the rotation matrix into penalty terms and adding them to the objective function. The specific form of the augmented Lagrangian function is: L = Σ||P_base_actual - (T_base_end_i * T_end_sensor * P_sensor_i)||^2 + λ||R^TR - I||^2 +μ||det(R) - 1||^2, where λ and μ are penalty coefficients, initially set to 1.0, and dynamically adjusted according to the degree of constraint violation during the iteration process, multiplied by a growth factor of 1.5 after each iteration.

[0087] The process of solving the augmented Lagrangian function includes: using gradient descent to locally optimize the current candidate parameters, calculating the gradient of the objective function with respect to the translation and rotation vectors, and updating the parameters along the gradient descent direction; checking the degree of constraint violation after each iteration, and increasing the penalty coefficient and continuing optimization if the constraints are not fully satisfied; repeating the above process until the constraints meet the accuracy requirements or the maximum number of local iterations is reached.

[0088] This embodiment effectively handles the orthogonality constraint of the rotation matrix during the optimization process, ensuring the physical rationality of the transformation matrix. At the same time, by dynamically adjusting the penalty coefficient, the relationship between minimizing the objective function and satisfying the constraints is balanced, thereby improving the accuracy and reliability of the calibration results.

[0089] Furthermore, the quality detection method also includes: if the fitness value of the current generation's best particle is not better than the global optimum, then calculate the fitness value of the current generation's worst particle; if the fitness value of the worst particle is better than the current global optimum, then determine that the particle state satisfies the convergence condition.

[0090] Specifically, the convergence condition judgment process includes: in each iteration of the particle swarm optimization algorithm, firstly, comparing the fitness value of the current generation's best particle with the global optimum; if the fitness value of the current generation's best particle is better than the global optimum, then updating the global optimum and continuing the iteration; if the fitness value of the current generation's best particle is not better than the global optimum, then further calculating the fitness value of the worst particle among all particles in the current generation (i.e., the particle with the largest objective function value); if the fitness value of the worst particle is still better than the current global optimum, then determining that the particle state has fully converged, satisfying the convergence condition, terminating the iteration, and outputting the current global optimum solution.

[0091] For example, if the global optimum is F_best = 0.05, and the fitness of the current best particle is F_current_best = 0.051, which is not better than the global optimum, then the fitness of the current worst particle is calculated as F_worst = 0.049, which is better than F_best = 0.05. At this point, the particle state is determined to meet the convergence condition, and the iteration stops.

[0092] Through this embodiment, this application provides an efficient early stopping strategy that can terminate the optimization process in a timely manner when the overall solution quality of the particle swarm is generally high, avoiding unnecessary iterative calculations, improving the efficiency of the calibration process, and ensuring the accuracy of the calibration results.

[0093] It should be noted that the target features described in this application include at least one of holes, edges, and planar geometric features.

[0094] See Figure 4 , Figure 4 This is a schematic diagram of the process for extracting the geometric parameters when the target feature is a hole, as described in an embodiment of this application.

[0095] As a specific embodiment of the present invention, when the target feature is a hole, the step of extracting the geometric parameters includes: S401: Perform multi-level plane fitting and projection filtering on the point cloud data to obtain the denoised hole point cloud.

[0096] The multi-level plane fitting and projection filtering are used to remove noise points and background interference from the point cloud, while preserving the accurate point cloud of the hole and its surrounding area. The multi-level plane fitting adopts a coarse-to-fine approach, first removing obvious outliers based on a larger distance threshold, and then accurately locating the plane where the hole is located based on a smaller distance threshold, effectively improving the point cloud quality.

[0097] For example, for a workpiece point cloud containing hole features, a first distance threshold of 5mm is set for coarse plane fitting to remove obvious noise and non-target plane points; then a second distance threshold of 2mm is set for fine plane fitting, and the retained points are projected onto the fine plane to obtain the denoised hole area point cloud.

[0098] S402: Perform curvature calculation and spatial continuity analysis on the denoised hole point cloud to screen out candidate points at the hole edges.

[0099] The curvature calculation is used to identify edge regions with large curvature in the point cloud, and the spatial continuity analysis is used to eliminate depth jump points caused by occlusion. The curvature calculation is based on the k-nearest neighbor to construct a local surface, and the principal curvature is calculated to reflect the degree of bending of the point; the spatial continuity analysis effectively identifies and eliminates false edge points caused by viewpoint occlusion by comparing the depth difference between adjacent points.

[0100] For example, set the k-nearest neighbor parameter k=30 and calculate the curvature value of each point; set the depth jump threshold to 10mm, and when the depth difference between adjacent points exceeds this threshold, it is determined to be a depth jump point; set the curvature threshold to 0.05, and filter out points with curvature greater than this threshold as candidate points for hole edges.

[0101] S403: Perform ellipse or circle fitting on the selected candidate points at the edge of the hole, and use the fitting center as the coordinates of the hole center.

[0102] In this process, the selected candidate points on the edge of the hole are fitted with an ellipse or a circle, and the fitted center is used as the coordinate of the hole center.

[0103] The fitting process involves selecting either an ellipse or a circle based on the actual shape of the hole. The center coordinates obtained from the fitting are used as the hole center coordinates, and the major and minor axes of the fitted ellipse or the diameter of the fitted circle are used as the hole diameter. The fitting process employs the least squares method, assigning different weights to different points through iterative weighting to improve fitting accuracy and robustness.

[0104] For example, for a circular hole with a diameter of 20mm, after the above steps, the deviation between the fitted hole center coordinates and the true value is less than 0.1mm, and the diameter measurement error is less than 0.05mm. For an elliptical hole with a major axis of 30mm and a minor axis of 20mm, the fitted major axis error is less than 0.1mm, and the minor axis error is less than 0.08mm.

[0105] Through this embodiment, this application achieves high-precision and robust extraction of workpiece hole features, removes noise through multi-level planar filtering, accurately identifies hole edges by combining curvature analysis and spatial continuity analysis, and finally obtains accurate hole center coordinates and hole diameter dimensions through ellipse or circle fitting, providing a reliable data foundation for subsequent quality judgment.

[0106] See Figure 5 , Figure 5 This is a schematic diagram illustrating the process of performing multi-level planar fitting and projection filtering on the point cloud data to obtain a denoised hole point cloud, as described in an embodiment of this application.

[0107] Step S401: Performing multi-level planar fitting and projection filtering on the point cloud data to obtain the denoised hole point cloud includes the following steps: S501: Perform plane fitting on the point cloud data based on the first distance threshold to obtain the first reference plane.

[0108] The first distance threshold is used to control the leniency of the plane fitting, allowing for larger distance deviations to quickly remove obvious outliers and non-target area point clouds.

[0109] For example, for a workpiece point cloud containing hole features, a first distance threshold of 5mm is set, and the RANSAC algorithm is used for plane fitting to obtain a first reference plane that can cover the approximate area where the hole is located.

[0110] S502: Remove discrete points whose distance to the first reference plane is greater than the first distance threshold.

[0111] The culling operation is used to remove noise points, background points, and other interfering point clouds that do not clearly belong to the target plane.

[0112] For example, calculate the distance from each point in the point cloud to the first reference plane. If the distance is greater than 5mm, it is determined to be a discrete point and discarded. The point cloud data with a distance within 5mm is retained.

[0113] S503: Project the points retained after removal onto the second reference plane obtained by fitting based on the second distance threshold to obtain the denoised hole point cloud.

[0114] The second distance threshold is less than the first distance threshold, which is used to more accurately locate the plane where the hole is located. The projection operation maps the retained point cloud data onto the accurately located plane to further remove noise.

[0115] For example, a second distance threshold of 2mm is set for the retained point cloud data, and the RANSAC algorithm is used again for plane fitting to obtain a second reference plane; then all the retained points are vertically projected onto the second reference plane to obtain the denoised hole region point cloud.

[0116] It should be noted that the first distance threshold is greater than the second distance threshold. For example, the first distance threshold is set to 5mm and the second distance threshold is set to 2mm. Through two-level plane fitting, which is first coarse and then fine, noise is effectively removed and the plane where the hole is located is accurately located.

[0117] Through this embodiment, this application realizes multi-level denoising processing of point cloud data. By adopting a coarse-to-fine plane fitting strategy, it not only ensures the denoising effect but also avoids the loss of hole feature information caused by over-filtering, thus providing high-quality point cloud data for subsequent hole edge extraction.

[0118] See Figure 6 . Figure 6 This is a schematic diagram illustrating the process of performing curvature calculation and spatial continuity analysis on the denoised hole point cloud to screen out candidate points at the hole edges, as described in an embodiment of this application.

[0119] Step S403: Performing curvature calculation and spatial continuity analysis on the denoised hole point cloud to screen out candidate points for hole edges includes the following steps: S601: Calculate the curvature value of each point in the denoised point cloud.

[0120] The curvature value is used to characterize the degree of curvature of the local surface where each point is located in the point cloud. The curvature value of the hole edge region is usually large due to the obvious curvature change.

[0121] For example, a local surface for each point is constructed based on the k-nearest neighbor algorithm, with k=30, and the principal curvature of each point is calculated to obtain the curvature value distribution that reflects the degree of bending of the point cloud.

[0122] S602: Analyze the continuity of depth between adjacent points in a point cloud sequence, and identify and eliminate depth jump points caused by occlusion.

[0123] The depth continuity analysis is used to exclude false edge points caused by viewpoint occlusion. These points have abrupt changes in the depth direction and do not belong to the real hole edges.

[0124] For example, traverse the point cloud sequence, calculate the difference between two adjacent points in the depth direction, set the depth jump threshold to 10mm, and when the depth difference between adjacent points exceeds this threshold, it is determined to be a depth jump point and excluded.

[0125] S603: Based on the curvature threshold, select points with larger curvature from non-depth transition points as candidate points for the edge of the hole.

[0126] The curvature threshold is used to distinguish between edge regions and flat regions, and points with curvature greater than the threshold are selected as candidate points for hole edges.

[0127] For example, by setting the curvature threshold to 0.05, points with a curvature greater than 0.05 can be selected from non-depth transition points as candidate points for hole edges. These points are located in the inner or outer edge regions of the hole.

[0128] Through this embodiment, this application achieves accurate identification of hole edge points, locates edge regions through curvature calculation, and eliminates occlusion interference by combining depth continuity analysis, effectively improving the screening accuracy of hole edge candidate points and providing reliable input data for subsequent ellipse or circle fitting.

[0129] In a specific embodiment of the present invention, when the target feature is an edge, the geometric parameter is the edge spacing.

[0130] See Figure 7 , Figure 7 This is a schematic diagram of the process for extracting edge spacing as described in an embodiment of this application.

[0131] The step of extracting the edge spacing includes the following steps: S701: Separate the point sets of the two sides from the point cloud data.

[0132] The separation operation is based on a region growing algorithm or a plane segmentation algorithm, which clusters the points in the point cloud data that belong to the two edges respectively, to obtain two independent point sets.

[0133] For example, for a workpiece point cloud containing parallel edges, a region growing algorithm based on normal vectors is used to cluster points with similar normal vectors and separate them into point sets for the two sides.

[0134] S702: Fitting a reference line based on a set of points on one side of the edge.

[0135] The reference line is fitted using the RANSAC algorithm or the least squares method, and the line equation is obtained by fitting from the set of edge points on one side.

[0136] For example, by selecting the left edge point set as the reference, the RANSAC algorithm is used to fit a straight line to obtain the slope and intercept parameters of the reference line.

[0137] S703: Calculate the distance from each point in the edge point set on the other side to the reference line.

[0138] The distance calculation is based on the point-to-line distance formula, which projects each point in the right edge point set onto the reference line to calculate the vertical distance.

[0139] For example, for each point (x_i, y_i) in the right edge point set, calculate its distance to the reference line ax + by + c = 0, d_i = |ax_i + by_i + c| / sqrt(a² + b²).

[0140] S704: Remove outliers from the calculated distance set based on the 3σ principle of normal distribution.

[0141] The 3σ principle is used to identify and eliminate outliers in the distance distribution, thereby improving the stability and reliability of the measurement results.

[0142] For example, calculate the mean μ and standard deviation σ of all distances, remove points whose distance values ​​are not in the interval [μ-3σ, μ+3σ], and retain the valid distance points.

[0143] S705: Calculate the average distance as the edge spacing based on the effective distance after rejection.

[0144] The average distance is the arithmetic mean of the remaining distances after removing outliers, and is used as the final distance measurement value of the two sides.

[0145] For example, for the effective distance set {d_1, d_2, ..., d_n}, the average distance D = (d_1 + d_2 + ... + d_n) / n is calculated as the edge spacing.

[0146] This embodiment achieves high-precision measurement of workpiece edge spacing. By using reference line fitting and distance calculation, combined with a 3σ outlier elimination strategy, the stability and anti-interference ability of the measurement results are effectively improved, providing an accurate basis for judging workpiece size and quality.

[0147] This application discloses an apparatus for using the described material delivery method. (See also...) Figure 8 The device includes a memory 311 configured to store various programs and data required for implementing the verification method. The memory 311 may be volatile memory or non-volatile memory, or may include both.

[0148] The memory 311 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 311 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0149] The memory 311 in this embodiment is used to store various types of data to support the operation of the device. Examples of this data include any computer programs used to operate on the device, such as operating systems and applications. The operating system includes various system programs, such as framework layers, core library layers, and driver layers, used to implement various basic business functions and handle hardware-based tasks. Applications can include various applications used to implement various application services. Here, the program implementing the method of this embodiment can be included in the application.

[0150] Based on the same inventive concept as the foregoing embodiments, this embodiment also provides a computer-readable storage medium storing a computer program. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, and airborne avionics systems of aircraft. When the computer program stored in the computer-readable storage medium is run by a processor, it implements the above method. For the specific steps of the computer program being executed by the processor, please refer to [link to relevant documentation]. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0151] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0152] In this document, the terms “comprising,” “including,” or any other variations thereof are intended to cover non-exclusive inclusion, which includes not only the elements listed but also other elements not expressly listed.

[0153] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A quality inspection method applied to robots, characterized in that, Obtain the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system; Based on the transformation matrix, the three-dimensional point cloud data obtained from scanning the workpiece under test is uniformly transformed to the base coordinate system; In the base coordinate system, the point cloud data is processed to extract the geometric parameters of the target features of the workpiece; The quality of the workpiece is determined based on the comparison results between the geometric parameters of the target feature and the preset standard.

2. The quality inspection method according to claim 1, characterized in that, Obtaining the transformation matrix from the preset sensor coordinate system to the robot end effector coordinate system includes the following steps: The robot is controlled to carry sensors to scan a fixed calibration object in multiple different postures, and to collect multiple sets of data simultaneously. Each set of data includes a frame of three-dimensional point cloud and the robot pose of the robot end in the base coordinate system when the three-dimensional point cloud is collected. Extract the feature point coordinates of the calibrated object in the preset sensor coordinate system from each frame point cloud; For each set of data, a set of theoretical coordinates for the calibration object is determined based on the coordinates of the feature points and the corresponding robot pose. Based on multiple sets of theoretical coordinates, construct an optimization problem for the transformation matrix; The transformation matrix is ​​obtained by solving the optimization problem using an optimization algorithm.

3. The quality inspection method according to claim 2, characterized in that, Solving the optimization problem using an optimization algorithm to obtain the transformation matrix includes the following steps: Candidate parameters for the transformation matrix are generated using a particle swarm optimization algorithm. In each iteration of the particle swarm optimization algorithm, the augmented Lagrange method is used to locally optimize the current candidate parameters; The particle state is updated based on the results of local optimization until the particle state satisfies the convergence condition, and then the transformation matrix is ​​output.

4. The quality inspection method according to claim 3, characterized in that, The step of using the augmented Lagrange method to locally optimize the current candidate parameters includes: An augmented Lagrangian function is constructed based on minimizing the deviation between the theoretical and actual coordinates of the feature points of the calibration object under different poses and the orthogonality of the rotation matrix in the transformation matrix. By solving the augmented Lagrangian function, local optimization of the candidate parameters can be achieved.

5. The quality inspection method according to claim 3, characterized in that, The method further includes: If the fitness value of the current generation's best particle is not better than the global best value, then calculate the fitness value of the current generation's worst particle. If the worst particle fitness value is better than the current global optimum, then the particle state is determined to satisfy the convergence condition.

6. The quality inspection method according to claim 1, characterized in that, The target feature includes a hole; When the target feature is a hole, the steps for extracting the geometric parameters include: Multi-level planar fitting and projection filtering are performed on the point cloud data to obtain a denoised hole point cloud. Curvature calculation and spatial continuity analysis are performed on the denoised hole point cloud to screen out candidate points at the hole edges; The selected candidate points at the edge of the hole are fitted with an ellipse or a circle, and the fitted center is used as the coordinate of the hole center.

7. The quality inspection method according to claim 6, characterized in that, The process of performing multi-level planar fitting and projection filtering on the point cloud data to obtain the denoised hole point cloud includes the following steps: Based on a first distance threshold, a first reference plane is obtained by performing plane fitting on the point cloud data. Discrete points whose distance to the first reference plane is greater than the first distance threshold are removed; The points retained after removal are projected onto a second reference plane fitted based on a second distance threshold to obtain the denoised hole point cloud; Wherein, the first distance threshold is greater than the second distance threshold.

8. The quality inspection method according to claim 6, characterized in that, The process of calculating the curvature and performing spatial continuity analysis on the denoised hole point cloud to screen out candidate points at the hole edges includes the following steps: Calculate the curvature value of each point in the denoised point cloud; Analyze the continuity of depth between adjacent points in a point cloud sequence, and identify and eliminate depth jump points caused by occlusion; Based on the curvature threshold, points with larger curvature are selected from non-depth transition points as candidate points for the hole edge.

9. The quality inspection method according to claim 1, characterized in that, The target feature includes edges, and the geometric parameter is the edge spacing; The steps for extracting the edge spacing include: The point sets of the two sides are separated from the point cloud data; Fit a reference line based on the set of points on one side of the edge; Calculate the distance from each point in the edge point set on the other side to the reference line; Outlier removal is performed on the calculated distance set based on the 3σ principle of normal distribution; The average distance after removing the effective distances is calculated as the edge spacing.

10. A device, characterized in that, The quality testing method according to any one of claims 1 to 9 shall be applied.

11. A computer-readable storage medium, characterized in that, The device contains a computer program that, when executed by a processor, implements the steps of the quality inspection method according to any one of claims 1 to 9.

Images