Error correction method, device and storage medium of visual measurement system

Abstract

The application discloses an error correction method and device of a visual measurement system and a storage medium. The method comprises the following steps: acquiring sensing data of a calibration object, extracting points corresponding to the calibration object based on the sensing data to obtain an actual point set; solving a mapping relationship between points in a theoretical point set of the calibration object and points in the actual point set based on the coordinates of the points in the theoretical point set and the coordinates of the points in the actual point set to obtain an affine transformation matrix; performing parameter decomposition on the affine transformation matrix to obtain a plurality of deformation variables, and calculating the contribution of each deformation variable to the error of the visual measurement system; selecting deformation variables based on the contribution of each deformation variable, selecting part or all of the deformation variables as correction parameters, and correcting the error of the visual measurement system. The method can avoid false adjustment caused by coupling interference between different error sources, realize quantitative correction of system error, improve the accuracy of error correction, and is high in process automation and parameter interpretability.

Description

Technical Field

[0001] This application relates to the field of computer vision measurement technology, and in particular to an error correction method, device and storage medium for a vision measurement system. Background Technology

[0002] Visual measurement cameras are widely used in automotive, electronics, and robotics fields due to their advantages such as high precision and non-contact operation. However, visual measurement errors are one of the key bottlenecks restricting the improvement of their accuracy. Among them, the angular errors in the XYZ directions (such as roll, pitch, and yaw) during camera installation are the most common sources of error. These angular errors are generally caused by mechanical installation gaps, stress deformation, dynamic vibration, etc., which can lead to image tilting and misalignment. Furthermore, the coupling of multiple directions and multiple error sources further aggravates the measurement deviation.

[0003] Existing calibration techniques have significant limitations, such as low efficiency of manual adjustment, high difficulty in calibration, high dependence on sensors such as levels, and the ability to handle errors in only one direction, making it difficult to meet high-precision requirements. Summary of the Invention

[0004] To address the aforementioned technical problems, this application provides at least one error correction method, device, and storage medium for a vision measurement system.

[0005] The first aspect of this application provides an error correction method for a vision measurement system. The method includes: acquiring perceptual data collected by the vision measurement system from a calibration object; extracting points corresponding to the calibration object based on the perceptual data to obtain a set of actual points; solving the mapping relationship between the points in the theoretical point set and the points in the actual point set based on the coordinates of each point in the preset theoretical point set of the calibration object and the coordinates of each point in the actual point set to obtain an affine transformation matrix; performing parametric decomposition on the affine transformation matrix to obtain various deformation variables, calculating the contribution of each deformation variable to the error of the vision measurement system; selecting deformation variables based on the contribution of each deformation variable, selecting some or all deformation variables as correction parameters, and correcting the error of the vision measurement system.

[0006] In one embodiment, based on the coordinates of each point in a predetermined theoretical point set and the coordinates of each point in a predetermined actual point set, the mapping relationship between the points in the theoretical point set and the points in the actual point set is solved to obtain an affine transformation matrix, including: A predetermined number of related points are selected from both the theoretical and actual point sets to obtain multiple pairs of related points. A system of linear equations is established based on the coordinates of these pairs, and the initial affine transformation matrix is ​​obtained by solving the system of linear equations. The initial affine transformation matrix is ​​applied to the actual point set to obtain the transformed point set, and the positional deviation between the points in the transformed point set and the points in the theoretical point set is calculated. Points in the actual point set whose positional deviation does not meet the predetermined deviation condition are removed, resulting in a removed set of actual points. The final affine transformation matrix is ​​obtained by resolving the system of equations based on the removed set of actual points and the theoretical point set.

[0007] In one embodiment, removing points from the actual point set whose position deviation does not meet the preset deviation condition to obtain a removed actual point set includes: calculating the standard deviation based on the position deviation of each point in the actual point set, calculating a screening threshold based on the standard deviation; removing points from the actual point set whose absolute value of the difference between the position deviation and the standard deviation is greater than the screening threshold to obtain a removed actual point set.

[0008] In one embodiment, parametric decomposition of the affine transformation matrix yields various deformations, including: parametric decomposition of the affine transformation matrix to obtain rotation matrix, scaling matrix, shearing matrix, and translation matrix.

[0009] In one embodiment, deformation variables are selected based on the contribution of each deformation variable, and some or all deformation variables are selected as correction parameters to perform error correction on the visual measurement system. This includes: statistically analyzing the numerical distribution of the contribution of each deformation variable, determining a deformation variable selection strategy based on the numerical distribution, and selecting some or all deformation variables as correction parameters based on the deformation variable selection strategy to perform error correction on the visual measurement system.

[0010] In one embodiment, calculating the contribution of each deformation variable to the error of the visual measurement system includes: obtaining the values ​​of the deformation variable in different coordinate system directions to obtain multiple direction values ​​corresponding to the deformation variable; obtaining the contribution of the deformation variable in different coordinate system directions based on the ratio between the multiple direction values ​​and the deformation variable; statistically analyzing the numerical distribution of the contribution of each deformation variable, and determining a deformation variable selection strategy based on the numerical distribution, including: based on the numerical distribution of the contribution of each deformation variable in different coordinate system directions, determining a deformation variable selection strategy of selecting one or more coordinate system directions whose contribution satisfies preset conditions to obtain a target direction, and using the direction values ​​corresponding to one or more deformation variables in the target direction as correction parameters.

[0011] In one embodiment, calculating the contribution of each deformation variable to the error of the visual measurement system includes: calculating the positional deviation between the points in the actual point set after each deformation variable transformation and the points in the theoretical point set, obtaining the component error corresponding to each deformation variable; calculating the contribution of each deformation variable based on the component error corresponding to each deformation variable; statistically analyzing the numerical distribution of the contribution of each deformation variable, and determining the deformation variable selection strategy based on the numerical distribution, including: if there are deformation variables with a contribution greater than a preset maximum threshold, the deformation variable selection strategy is to select the deformation variables with a contribution greater than the preset maximum threshold as correction parameters; and / or, if there are no deformation variables with a contribution greater than the preset maximum threshold, and there are deformation variables with a contribution less than a preset minimum threshold, the deformation variable selection strategy is to remove the deformation variables with a contribution less than the preset minimum threshold and use the remaining deformation variables as correction parameters; and / or, if the difference between the contributions of each deformation variable is less than a preset difference threshold, the deformation variable selection strategy is to select all deformation variables as correction parameters.

[0012] In one embodiment, the method further includes: calculating the positional deviation between the points in the actual point set after transformation by the affine transformation matrix and the points in the theoretical point set to obtain the total error; if the total error is greater than a preset error threshold, then the physical position of the visual measurement system is corrected for error; if the total error is not greater than the preset error threshold, then the perceived data of the visual measurement system is corrected for error.

[0013] The second aspect of this application provides an error correction device for a visual measurement system. The device includes: a real point extraction module, used to acquire perceptual data collected by the visual measurement system from a calibration object, and extract points corresponding to the calibration object based on the perceptual data to obtain a set of real points; an affine transformation solving module, used to solve the mapping relationship between points in the theoretical point set and points in the real point set based on the coordinates of each point in the preset theoretical point set of the calibration object and the coordinates of each point in the real point set to obtain an affine transformation matrix; a contribution calculation and extraction module, used to perform parametric decomposition on the affine transformation matrix to obtain various deformation variables, and calculate the contribution of each deformation variable to the error of the visual measurement system; and an error correction module, used to select deformation variables based on the contribution of each deformation variable, select some or all deformation variables as correction parameters, and perform error correction on the visual measurement system.

[0014] A third aspect of this application provides an electronic device, including a memory and a processor, wherein the processor is configured to execute program instructions stored in the memory to implement the error correction method of the aforementioned vision measurement system.

[0015] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the error correction method of the aforementioned vision measurement system.

[0016] The above scheme acquires perceptual data collected by the vision measurement system from the calibration object, extracts the corresponding points of the calibration object based on the perceptual data, and obtains the actual point set; based on the coordinates of each point in the theoretical point set of the calibration object and the coordinates of each point in the actual point set, it solves the mapping relationship between the points in the theoretical point set and the points in the actual point set, and obtains the affine transformation matrix; it performs parametric decomposition on the affine transformation matrix to obtain various deformation variables, calculates the contribution of each deformation variable to the error of the vision measurement system; based on the contribution of each deformation variable, it selects deformation variables, selects some or all deformation variables as correction parameters, and performs error correction on the vision measurement system, avoiding erroneous adjustments caused by coupling interference between different error sources, realizing quantitative correction of system error, reducing system adjustment costs, improving the accuracy of error correction, and the process is automated with high parameter interpretability.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0019] Figure 1 This is a flowchart illustrating an error correction method for a vision measurement system, as shown in an exemplary embodiment of this application. Figure 2 This is a schematic diagram showing a beveled section, illustrating an exemplary embodiment of this application; Figure 3 This is a schematic diagram of rotation shown in an exemplary embodiment of this application; Figure 4 This is a block diagram illustrating an error correction device for a vision measurement system, as shown in an exemplary embodiment of this application. Figure 5 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of this application; Figure 6 This is a schematic diagram illustrating the structure of a computer-readable storage medium, as shown in an exemplary embodiment of this application. Detailed Implementation

[0020] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0021] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0022] In this document, the term "and / or" is merely a description of the association information of related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. Moreover, the term "at least one" in this document means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.

[0023] The error correction method for the vision measurement system provided in the embodiments of this application will be described below.

[0024] The error correction method for the visual measurement system in this application can be executed by the visual measurement system itself, other electronic devices that are connected in communication with the visual measurement system, or by the visual measurement system and other electronic devices working together. This application does not limit the scope of the method.

[0025] Please see Figure 1 , Figure 1 This is a flowchart illustrating an error correction method for a vision measurement system according to an exemplary embodiment of this application, as shown below. Figure 1 As shown, the error correction method for the vision measurement system includes at least steps S110 to S140, which are described in detail below: Step S110: Obtain the perception data collected by the vision measurement system on the calibration object, extract the points corresponding to the calibration object based on the perception data, and obtain the actual point set.

[0026] Visual measurement systems utilize cameras, image sensors, and computer vision algorithms to measure the geometric parameters of objects, such as size, shape, position, and distance.

[0027] The visual measurement system may be based on a visualization camera, and / or a 3D (Dimensional) line laser camera, and / or a structured light camera, and / or a binocular camera for environmental perception, and this application does not limit it.

[0028] The visual measurement system collects data from the calibration object to obtain the perceived data of the calibration object.

[0029] The calibrator can be a frustum, and / or a boss, and / or a calibrator sphere, and / or a spherical array, and / or a frustum array, etc. This application does not limit the type of calibrator.

[0030] Perceptual data can be visualized images, and / or point cloud images, and / or depth images, etc. This application does not limit the data type of perceptual data.

[0031] Then, points corresponding to the calibration object are extracted based on the perception data. The extracted points may include real points on the calibration object, such as corner points and edge contour points of the calibration object, or virtual points corresponding to the calibration object, such as virtual center points and centroid points of the calibration object. This application does not limit the type of points.

[0032] Taking a truncated pyramid as the calibration object and a visual measurement system including a 3D line laser camera as an example: After the 3D line laser camera acquires the depth data of the truncated pyramid, the first-order gradient is used to calculate the gradient in the XY directions of the truncated pyramid image, pre-separating the front, back, left, right, middle, and bottom areas to obtain segmented regions; morphological closing operations are performed on the segmented regions to fill internal holes and gaps, resulting in filled regions; to avoid the influence of boundary regions, the boundaries of each filled region are scaled inward to obtain scaled regions; plane fitting is performed on each scaled region, with the fitting formula ax + by + cz + d = 0, yielding the plane equations for each plane, where (x, y, z) are the coordinates of any point, and a, b, c, and d are real coefficients. Since the three intersecting planes have a unique corner point, the vertices are calculated using the least squares solution of the intersection points of the three planes, and the vertices are added to the actual point set P.

[0033] Taking a stepped spherical array and a vision measurement system including a 3D line laser camera as an example: The 3D line laser camera acquires the depth data of all stepped spherical arrays, and uses the first-order gradient to calculate the gradient in the XY direction of each stepped spherical array image to segment the individual spherical regions, thus obtaining segmented regions; spherical fitting is performed on the segmented regions corresponding to all spheres, and the fitting formula is (x-x0)^2+(y-y0)^2+(z-z0)^2=R^2, where (x,y,z) are the coordinates of any point, and R is the radius of the sphere, thus obtaining the center point (x0,y0,z0), and adding the center point to the actual point set P.

[0034] Step S120: Based on the coordinates of each point in the theoretical point set and the coordinates of each point in the actual point set, solve the mapping relationship between the points in the theoretical point set and the points in the actual point set to obtain the affine transformation matrix.

[0035] Obtain the prior physical location of the calibration object to obtain the theoretical point set.

[0036] Based on the coordinates of each point in the theoretical point set Q and the coordinates of each point in the actual point set P, a mapping model between theoretical and actual points is established, and the transformation relationship is solved to obtain the affine transformation matrix.

[0037] Taking a vision measurement system including a 3D line laser camera as an example: When installing a 3D line laser camera, the forward direction is generally required to be perpendicular to the light plane. However, in actual production environments, due to factors such as installation angle errors and workpiece stress deformation, varying degrees of angular errors may occur in the XYZ directions, indirectly affecting the point cloud quality. For example, when the camera rotates around the X-axis, it will cause the scanned point cloud to be obliquely cut in the YZ plane (causing a perfect circle to become an ellipse, or a right angle to become an acute or obtuse angle). Please refer to [link to relevant documentation]. Figure 2 , Figure 2 This is a schematic diagram of a beveled section illustrating an exemplary embodiment of this application, as shown below. Figure 2 As shown, oblique cutting causes right angles within the rectangle to deform into acute or obtuse angles; when the camera rotates around the Y-axis, the light plane is perpendicular to the forward direction, and the point cloud does not deform, but it will rotate around the Y-axis, as shown in the following figure. Figure 3 , Figure 3 This is a schematic diagram of rotation shown in an exemplary embodiment of this application, as follows: Figure 3 As shown, rotation causes a deviation in the position of the point cloud; when the camera rotates around the Z-axis, a slant appears on the XY plane; when the error angles in the above multiple directions are superimposed, a complex affine deformation will be produced, that is, the generation of system error is a 3D affine transformation.

[0038] Step S130: Perform parametric decomposition on the affine transformation matrix to obtain various deformation variables, and calculate the contribution of each deformation variable to the error of the visual measurement system.

[0039] An affine transformation matrix is ​​the result of a sequential combination of multiple basic transformations (such as rotation, and / or scaling, and / or shearing, and / or translation). By performing parametric decomposition on the affine transformation matrix, it can be decomposed into various shapes and variables.

[0040] For example, parametric decomposition of an affine transformation matrix yields shape variables including rotation, scaling, shearing, and translation matrices.

[0041] Then, the contribution of each deformation to the error of the visual measurement system is calculated.

[0042] In a vision measurement system, the final output of the system is affected by multiple error sources. Different deformation variables are considered as different error sources. The contribution of the deformation variable can refer to the degree of influence of the deformation variable on the output error of the vision measurement system, that is, the proportion of the total error shared by each deformation variable; and / or the contribution of the deformation variable can also refer to the degree of error influence of the deformation variable in different coordinate system directions (such as the XYZ direction).

[0043] The contribution of deformation variables to the error of the visual measurement system can be determined by numerical analysis of the deformation variables; alternatively, a neural network model for contribution analysis can be pre-trained and used to determine the contribution of deformation variables to the error of the visual measurement system. The specific calculation method of the contribution can be flexibly selected according to the actual application scenario, and this application does not limit it.

[0044] Step S140: Select deformation variables based on the contribution of each deformation variable, select some or all deformation variables as correction parameters, and perform error correction on the visual measurement system.

[0045] After obtaining the contribution of each deformation variable, deformation variables are selected based on their contribution. Some or all of the selected deformation variables are used as correction parameters, and the visual measurement system is corrected for errors based on these correction parameters.

[0046] For example, a selection threshold is set for each type of deformation. The magnitude relationship between each type of deformation and its selection threshold is compared, and the deformation that meets the preset conditions is selected to obtain the correction parameter. The visual measurement system is then corrected for errors based on the correction parameter.

[0047] For example, the deformation includes the rotation matrix. If the rotation matrix is ​​greater than the selected threshold of 0.5°, it is determined that the rotation matrix meets the preset conditions, and the rotation matrix is ​​selected as the correction parameter for rotation correction.

[0048] The selected threshold can be a pre-set empirical value or a value that can be flexibly set according to the actual application scenario.

[0049] For example, based on the measurement task of the vision measurement system, the selection threshold for each deformation variable is determined. For instance, if the measurement task is dimensional measurement, the selection threshold for the scaling matrix is ​​set smaller; if the measurement task is shape detection, the selection threshold for the shearing matrix is ​​set smaller. By setting the selection threshold for each deformation variable according to the sensitivity of different measurement tasks to different error sources, task-adaptive threshold adjustment is achieved, improving the adaptability between error correction results and measurement tasks.

[0050] For example, in addition to selecting deformation variables by selecting a threshold, one can also directly select the deformation variable with the largest contribution, or sort the deformation variables in descending order according to their contribution and select the top N deformation variables to obtain the correction parameters.

[0051] For example, in addition to selecting the entire deformation as the correction parameter, the values ​​of the deformation in a specified coordinate system direction can also be selected, that is, a portion of the deformation values ​​can be selected to obtain the correction parameter.

[0052] The strategy for selecting deformation variables can be predetermined based on experience or flexibly set according to the actual situation.

[0053] For example, the shape selection strategy can be determined based on the positional deviation between the theoretical point set and the actual point set, or the positional deviation between the theoretical point set and the points after the actual point set has been transformed by the affine transformation matrix; and / or, the shape selection strategy can be determined based on the numerical distribution of the contribution of each shape variable, which is not limited in this application.

[0054] In addition to selecting deformation variables, the correction priority of the selected deformation variables can also be set. For example, the priority can be sorted directly according to the contribution, with the higher the contribution, the higher the priority. The error correction is then performed according to the selected deformation variables in turn according to the priority sorting result, so as to correct the error sources with greater impact first and improve the correction efficiency.

[0055] Optionally, the error correction process can be iteratively executed multiple times. The physical position of the visual measurement system is adjusted based on the selected correction parameters each time. After each adjustment, steps S110 to S140 are repeated, and the actual point set is re-acquired after each adjustment. This continues until the sum of the positional deviations between the theoretical and actual point sets, or the sum of the positional deviations between the theoretical and actual point sets after transformation using an affine transformation matrix, is less than a preset deviation threshold. This completes the error correction, and the accuracy is improved through multiple iterations. Furthermore, to ensure accuracy, the coordinates of the perceived data can be further corrected using the recalculated affine transformation matrix after error correction.

[0056] Alternatively, the selected deformation can be used as a physical correction parameter to adjust the physical position of the vision measurement system; the unselected deformation can be used as a software correction parameter to correct the coordinates of the perception data collected by the vision measurement system. By combining physical correction and software correction, error correction can be achieved quickly.

[0057] This application uses affine transformation component decomposition to clarify the contribution ratio of each error source. Based on the contribution, it selects some or all deformation variables for error correction, avoiding over-adjustment caused by coupling interference between different error sources. This achieves quantitative correction of system errors, reduces system adjustment costs, improves the accuracy of error correction, reduces the impact of point cloud tilt, rotation, scaling, and skewing, automates the process, and improves parameter interpretability. It provides quantitative basis for camera installation and debugging, tooling and fixture optimization, etc., is applicable to multiple camera models, and can be quickly integrated into the production line.

[0058] The following describes some embodiments of this application in detail.

[0059] In some implementations, step S120 involves solving the mapping relationship between the points in the theoretical point set and the points in the actual point set based on the coordinates of each point in the theoretical point set and the coordinates of each point in the actual point set, to obtain the affine transformation matrix, including the following steps S121 to S125.

[0060] Step S121: Select a preset number of interrelated points from the theoretical point set and the actual point set respectively to obtain multiple pairs of related points.

[0061] Based on the correspondence between points in the theoretical point set and points in the actual point set, we determine the points that are related to each other, and select a preset number of related point pairs.

[0062] For example, if point q1 in the theoretical point set and point p1 in the actual point set are both the first point at the top left corner of the calibration object, then point q1 and point p1 are considered as a pair of related points.

[0063] The number of associated point pairs is determined by the dimension of the affine transformation. For example, at least 3 associated point pairs are required for a 2D affine transformation, and at least 4 associated point pairs are required for a 3D affine transformation.

[0064] Step S122: Establish a system of linear equations based on the coordinates of multiple related point pairs, and solve the system of linear equations to obtain the initial affine transformation matrix.

[0065] Affine transformation is a linear transformation (which can include rotation, scaling, translation, shearing, etc.) and is represented by a matrix in 2D or 3D space.

[0066] Taking 3D affine transformation as an example, the coordinates of the points in the actual point set of the associated point pair are represented as follows: The coordinates of the points in the theoretical point set are represented as: Then, the system of linear equations obtained can be expressed as Equation 1 below: (Formula 1) in, Let H be the matrix to be solved.

[0067] For each 3D point, three systems of equations are constructed, resulting in a total of 12 unknowns. At least four point pairs are required. After expanding the H matrix into 1D column vectors, the matrix equation system is constructed as shown in Formula 2 below: (Formula 2) Then, the overdetermined system of equations X=(A^TA)^{-1}A^TB is solved by the pseudo-inverse method to obtain the initial affine transformation matrix.

[0068] Step S123: Apply the initial affine transformation matrix to the actual point set to obtain the transformed point set, and calculate the positional deviation between the points in the transformed point set and the points in the theoretical point set.

[0069] For example, applying an initial affine transformation matrix to the actual set of points yields a transformed set of points, where any point in the transformed set is represented as... The corresponding associated points in the theoretical point set are represented as Then, the positional deviation of point i is calculated using the following formula 3. : (Formula 3) Step S124: Remove points from the actual point set whose positional deviation does not meet the preset deviation condition, and obtain the removed actual point set.

[0070] For example, the points in the actual point set can be sorted in ascending order based on the positional deviation, and a preset number of points at the top of the sort can be retained, while the remaining points can be removed to obtain the actual point set after removal.

[0071] For example, it could also involve obtaining a filtering threshold, removing points whose positional deviation is greater than the filtering threshold, and obtaining the set of actual points after removal.

[0072] For example, removing points from the actual point set whose position deviation does not meet the preset deviation condition to obtain the removed actual point set includes: calculating the standard deviation based on the position deviation of each point in the actual point set, calculating the screening threshold based on the standard deviation; removing points from the actual point set whose absolute value of the difference between the position deviation and the standard deviation is greater than the screening threshold to obtain the removed actual point set.

[0073] Specifically, the mean deviation is calculated based on the positional deviation of each point in the actual point set. Let n be the total number of points, and then calculate the standard deviation based on the mean deviation. The screening threshold is calculated based on the standard deviation.

[0074] For example, the standard deviation can be multiplied by a preset standard deviation factor to obtain the screening threshold. Of course, the standard deviation factor can also be flexibly determined according to the actual situation. For example, the standard deviation factor can be determined according to the type of measurement task and the accuracy requirements.

[0075] For example, if the standard deviation factor is 6, then points will be removed from the actual point set. The points are then used to obtain the set of actual points after removal.

[0076] By selecting points within the screening threshold range based on Gaussian distribution, points with obvious abnormal errors (mostly caused by image noise, surface defects of calibration objects, etc.) are removed, effectively reducing the impact of noise and calibration object defects on the correction results. After removing abnormal points, the model fitting degree is improved.

[0077] Step S125: Based on the removed set of actual points and the set of theoretical points, re-solve to obtain the final affine transformation matrix.

[0078] The affine transformation matrix H is solved by minimizing the objective function f, as detailed in Formula 4 below: (Formula 4) The gradient descent method is used to iteratively correct H, minimizing the value of the optimization objective function, and finally obtaining the optimized affine transformation matrix H_{opt}.

[0079] Then, the final optimized affine transformation matrix H_{opt} is decomposed parametrically to obtain various deformation variables.

[0080] For example, the 3×3 submatrix in the optimized affine transformation matrix H_{opt} is decomposed into a rotation matrix R, a scaling matrix S, and a shearing matrix H_s through QR decomposition. Then, combined with the translation matrix t, we get the decomposed H_{opt} = [R·S·H_s | t].

[0081] Then, based on the contribution of each deformation variable, deformation variables are selected, and some or all of the deformation variables are selected as correction parameters to correct the error of the visual measurement system.

[0082] In the above embodiments, the system error is modeled as a 3D affine transformation, which can realize integrated correction of rotation, scaling, shearing, mirroring and translation, thereby improving the correction accuracy.

[0083] For example, in step S140, based on the contribution of each deformation variable, some or all deformation variables are selected as correction parameters to perform error correction on the visual measurement system, including the following steps S141 to S142.

[0084] Step S141: Calculate the numerical distribution of the contribution of each deformation variable, and determine the deformation variable selection strategy based on the numerical distribution.

[0085] For example, calculating the contribution of each deformation variable to the error of the visual measurement system includes: obtaining the values ​​of the deformation variable in different coordinate system directions to obtain multiple direction values ​​corresponding to the deformation variable; obtaining the contribution of the deformation variable in different coordinate system directions based on the ratio between the multiple direction values ​​and the deformation variable; then statistically analyzing the numerical distribution of the contribution of each deformation variable, and determining the deformation variable selection strategy based on the numerical distribution, including: based on the numerical distribution of the contribution of each deformation variable in different coordinate system directions, the determined deformation variable selection strategy is to select one or more coordinate system directions whose contribution satisfies preset conditions to obtain the target direction, and using the direction values ​​corresponding to one or more deformation variables in the target direction as correction parameters.

[0086] Taking the deformation as the rotation matrix as an example, based on the values ​​of the rotation matrix in the X, Y, and Z directions of different coordinate systems, the contribution of the rotation matrix in the X direction is calculated to be 0.7, the contribution in the Y direction is 0.1, and the contribution in the Z direction is 0.2.

[0087] It can be done by summing the contribution of each deformation along the coordinate system direction and selecting the coordinate system direction with the largest total contribution or the total contribution greater than a preset threshold as the target direction; or it can be done by calculating the maximum contribution among the contributions of each deformation along the coordinate system direction and obtaining the maximum contribution, selecting the coordinate system direction with the maximum contribution greater than a preset threshold or the coordinate system direction with the largest maximum contribution as the target direction.

[0088] For example, if the maximum contribution in the X direction is 0.7 (the contribution of the rotation matrix), the maximum contribution in the Y direction is 0.6 (the contribution of the scaling matrix), and the maximum contribution in the Z direction is 0.3 (the contribution of the shearing matrix), and the preset contribution threshold is 0.5, then the X and Y directions are selected as the target directions.

[0089] Then, the direction values ​​corresponding to all deformations in the target direction, or the direction values ​​corresponding to deformations in the target direction whose contribution is greater than a preset threshold, are used as correction parameters.

[0090] For example, if the target direction includes the X direction, the direction values ​​corresponding to the rotation matrix, scaling matrix, shearing matrix, and translation matrix in the X direction can be used as correction parameters; alternatively, the deformations with a contribution greater than a preset threshold in the X direction can be identified as rotation matrices, and the direction values ​​of the rotation matrices in the X direction can be used as correction parameters.

[0091] The above embodiments decouple the deformation in different coordinate system directions to correct the coupling angle error in the XYZ three directions, avoid mutual interference between different coordinate system directions, and improve the accuracy of correction.

[0092] For example, calculating the contribution of each deformation variable to the error of the visual measurement system includes: calculating the positional deviation between the points in the actual point set after each deformation variable transformation and the points in the theoretical point set, obtaining the component error corresponding to each deformation variable; calculating the contribution of each deformation variable based on the component error corresponding to each deformation variable; then statistically analyzing the numerical distribution of the contribution of each deformation variable, and determining the deformation variable selection strategy based on the numerical distribution, including: if there are deformation variables with a contribution greater than a preset maximum threshold, the deformation variable selection strategy is to select the deformation variables with a contribution greater than the preset maximum threshold as correction parameters; and / or, if there are no deformation variables with a contribution greater than the preset maximum threshold, and there are deformation variables with a contribution less than a preset minimum threshold, the deformation variable selection strategy is to remove the deformation variables with a contribution less than the preset minimum threshold and use the remaining deformation variables as correction parameters; and / or, if the difference between the contributions of each deformation variable is less than a preset difference threshold, the deformation variable selection strategy is to select all deformation variables as correction parameters.

[0093] Specifically, if the positional deviation between the actual point set and the theoretical point set is smaller after the actual point set is transformed according to the deformation variable, then the deformation variable has a greater impact on the error of the visual measurement system, and the corresponding contribution of the deformation variable is greater; conversely, the larger the positional deviation, the smaller the deformation variable has impact on the error of the visual measurement system, and the corresponding contribution of the deformation variable is smaller.

[0094] Of course, in addition to measuring the degree of error influence by calculating the positional deviation obtained by transforming the actual point set based on a single deformation variable, a specified deformation variable can also be removed from each deformation variable, and then the actual point set can be transformed based on the remaining deformation variables to calculate the positional deviation. Similarly, the positional deviation is used to measure the degree of error influence of the specified deformation variable. The difference is that the larger the positional deviation, the greater the contribution of the specified deformation variable, and vice versa.

[0095] The specific calculation method for the contribution of each deformation variable can be flexibly selected according to the actual application scenario, and this application does not limit it.

[0096] In conjunction with the above embodiments, the following can be implemented: Variables with a contribution greater than a preset maximum threshold can be selected as correction parameters to prioritize error correction based on variables with a large error impact; and / or, if no variable has a contribution greater than the preset maximum threshold, but there are variables with a contribution less than the preset minimum threshold, then the variables with a contribution less than the preset minimum threshold are removed, and the remaining variables are used as correction parameters to ignore variables with a small error impact and avoid over-adjustment; and / or, if the difference between the contributions of each variable is less than a preset difference threshold, it indicates that the error impact of each variable is relatively average, and in this case, all variables are directly selected as correction parameters.

[0097] Of course, in addition to the deformation selection strategies exemplified above, other deformation selection strategies can also be adopted. For example, if the difference between the contributions of each deformation variable is greater than a preset difference threshold, the deformation variable with the largest contribution is selected as the correction parameter.

[0098] Step S142: Based on the deformation selection strategy, select some or all deformation variables as correction parameters to perform error correction on the visual measurement system.

[0099] Error correction can be achieved through physical correction and software correction. Physical correction refers to adjusting the physical position of the vision measurement system according to correction parameters; software correction refers to directly transforming the coordinates of the perception data collected by the vision measurement system according to correction parameters.

[0100] In some factual approaches, the method further includes: calculating the positional deviation between the points in the actual point set after transformation by the affine transformation matrix and the points in the theoretical point set to obtain the total error; if the total error is greater than a preset error threshold, then the physical position of the visual measurement system is corrected for error; if the total error is not greater than the preset error threshold, then the perceived data of the visual measurement system is corrected for error.

[0101] For example, the total error can be obtained by summing the positional deviations of each point calculated by Formula 3; or the average deviation can be calculated based on the positional deviations of each point and used as the total error. This application does not limit the specific calculation method of the total error.

[0102] If the total error exceeds the preset error threshold, it indicates that the current error of the visual measurement system is large. Error correction should be performed on the physical position of the visual measurement system (such as correcting the rotation angle, compensating for the translation, etc.) to quickly reduce the total error. If the total error is not greater than the preset error threshold, it indicates that the current error of the visual measurement system is small. Error correction can be performed directly on the perceived data of the visual measurement system. This can be done by transforming the perceived data of the visual measurement system according to each type of deformation obtained from the decomposition, or by selecting a portion of the deformations to transform the perceived data of the visual measurement system, or by directly transforming the perceived data of the visual measurement system according to the affine transformation matrix, thereby achieving error correction of the coordinates of the perceived data.

[0103] The above embodiments can achieve the correction of various deformations after decoupling. After correction, the root mean square error (RMSE) can be reduced from 1.2mm to 0.02mm, which meets the measurement requirements of precision manufacturing.

[0104] Figure 4 This is a block diagram illustrating an error correction device for a vision measurement system, as shown in an exemplary embodiment of this application. Figure 4 As shown, the error correction device 400 of this exemplary vision measurement system includes: The actual point extraction module 410 is used to acquire the perception data collected by the vision measurement system on the calibration object, and extract the points corresponding to the calibration object based on the perception data to obtain the actual point set; The affine transformation solution module 420 is used to solve the mapping relationship between the points in the theoretical point set and the points in the actual point set based on the coordinates of each point in the theoretical point set and the coordinates of each point in the actual point set preset by the calibration object, and obtain the affine transformation matrix. The contribution calculation and extraction module 430 is used to perform parametric decomposition on the affine transformation matrix to obtain various deformation variables, and to calculate the contribution of each deformation variable to the error of the visual measurement system. Error correction module 440 is used to select deformation variables based on the contribution of each deformation variable, select some or all deformation variables as correction parameters, and perform error correction on the visual measurement system.

[0105] It should be noted that the error correction device for the vision measurement system provided in the above embodiments and the error correction method for the vision measurement system provided in the above embodiments belong to the same concept. The specific operation methods of each module and unit have been described in detail in the method embodiments and will not be repeated here. In practical applications, the error correction device for the vision measurement system provided in the above embodiments can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. This is not a limitation.

[0106] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment of this application. The electronic device 500 includes a memory 510 and a processor 520. The processor 520 executes program instructions stored in the memory 510 to implement the steps in any of the above-described embodiments of the error correction method for the vision measurement system. In a specific implementation scenario, the electronic device 500 may include, but is not limited to, a microcomputer or a server. Furthermore, the electronic device 500 may also include mobile devices such as laptops and tablets, without limitation.

[0107] Specifically, processor 520 controls itself and memory 510 to implement the steps in any of the above-described error correction method embodiments of the vision measurement system. Processor 520 may also be referred to as a Central Processing Unit (CPU). Processor 520 may be an integrated circuit chip with signal processing capabilities. Processor 520 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 520 may be implemented using integrated circuit chips.

[0108] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the structure of a computer-readable storage medium according to an exemplary embodiment of this application. The computer-readable storage medium 600 stores program instructions 610 that can be executed by a processor. The program instructions 610 are used to implement the steps in any of the above-described embodiments of the error correction method for the vision measurement system.

[0109] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0110] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0111] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A method of error correction for a vision measurement system, characterized by, The method includes: Acquire the perception data collected by the vision measurement system on the calibration object, and extract the points corresponding to the calibration object based on the perception data to obtain the actual point set; Based on the coordinates of each point in the theoretical point set preset by the calibration object and the coordinates of each point in the actual point set, the mapping relationship between the points in the theoretical point set and the points in the actual point set is solved to obtain the affine transformation matrix. The affine transformation matrix is ​​parametrically decomposed to obtain various deformation variables, and the contribution of each deformation variable to the error of the vision measurement system is calculated. Based on the contribution of each deformation variable, deformation variables are selected, and some or all of the deformation variables are selected as correction parameters to perform error correction on the visual measurement system.

2. The method of claim 1, wherein, The mapping relationship between the points in the theoretical point set and the points in the actual point set, based on the coordinates of each point in the preset theoretical point set and the coordinates of each point in the actual point set, is solved to obtain the affine transformation matrix, including: A predetermined number of interrelated points are selected from the theoretical point set and the actual point set respectively to obtain multiple pairs of related points; A system of linear equations is established based on the coordinates of the multiple associated point pairs, and the initial affine transformation matrix is ​​obtained by solving the system of linear equations. Apply the initial affine transformation matrix to the actual point set to obtain the transformed point set, and calculate the positional deviation between the points in the transformed point set and the points in the theoretical point set; Points whose positional deviations do not meet the preset deviation conditions are removed from the actual point set to obtain the removed actual point set; The final affine transformation matrix is ​​obtained by resolving the actual point set and the theoretical point set after the removal.

3. The method of claim 2, wherein, The step of removing points from the actual point set whose positional deviations do not meet the preset deviation conditions, to obtain the removed actual point set, includes: The standard deviation is calculated based on the positional deviation of each point in the actual point set, and the screening threshold is calculated based on the standard deviation. Points in the actual point set whose absolute difference between the position deviation and the standard deviation is greater than the filtering threshold are removed, resulting in the removed actual point set.

4. The method of claim 1, wherein, The parametric decomposition of the affine transformation matrix yields various deformation variables, including: The affine transformation matrix is ​​decomposed parametrically to obtain the rotation matrix, scaling matrix, shearing matrix, and translation matrix.

5. The method of claim 1, wherein, The process of selecting deformation variables based on the contribution of each deformation variable, and selecting some or all deformation variables as correction parameters to perform error correction on the visual measurement system includes: The numerical distribution of the contribution of each type of deformation variable is statistically analyzed, and a deformation variable selection strategy is determined based on the numerical distribution. Based on the aforementioned deformation selection strategy, some or all deformation variables are selected as correction parameters to perform error correction on the visual measurement system.

6. The method of claim 5, wherein, The calculation of the contribution of each deformation to the error of the visual measurement system includes: The values ​​of the deformation in different coordinate systems are obtained to obtain the values ​​of the deformation in multiple directions. Based on the ratio between the multiple directional values ​​and the deformation, the contribution of the deformation in different coordinate system directions is obtained; The step of statistically analyzing the numerical distribution of the contribution of each deformable variable, and determining the deformable variable selection strategy based on the numerical distribution, includes: Based on the numerical distribution of the contribution of each deformation variable in different coordinate system directions, the determined deformation variable selection strategy is to select one or more coordinate system directions whose contribution satisfies the preset conditions to obtain the target direction, and use the direction values ​​corresponding to one or more deformation variables in the target direction as correction parameters.

7. The method of claim 5, wherein, The calculation of the contribution of each deformation to the error of the visual measurement system includes: Calculate the positional deviation between the points in the actual point set after each of the deformation transformations and the points in the theoretical point set, and obtain the component error corresponding to each deformation. Based on the component error corresponding to each deformation variable, the contribution degree corresponding to each deformation variable is calculated. The step of statistically analyzing the numerical distribution of the contribution of each deformable variable, and determining the deformable variable selection strategy based on the numerical distribution, includes: If there are deformable variables whose contribution is greater than the preset maximum threshold, the deformable variable selection strategy is to select the deformable variables whose contribution is greater than the preset maximum threshold as correction parameters; And / or, if there are no deformation variables with a contribution greater than the preset maximum threshold, and there are deformation variables with a contribution less than the preset minimum threshold, then the deformation variable selection strategy is to remove the deformation variables with a contribution less than the preset minimum threshold and use the remaining deformation variables as correction parameters. And / or, if the difference between the contributions of each deformation variable is less than a preset difference threshold, the deformation variable selection strategy is to select all deformation variables as correction parameters.

8. The method of claim 1, wherein, The method further includes: Calculate the positional deviation between the points in the actual point set after transformation by the affine transformation matrix and the points in the theoretical point set to obtain the total error; If the total error is greater than a preset error threshold, then the physical position of the vision measurement system is corrected for error. If the total error is not greater than a preset error threshold, then the perception data of the vision measurement system is corrected for error.

9. An electronic device, comprising: The electronic device includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the steps of the method as described in any one of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor to implement the steps of the method as described in any one of claims 1-8.

Images