A detection method based on mobile device and vision sensor
By pre-calibrating and calculating the pose change matrix of the vision sensor, the problems of unstable calibration accuracy and online detection of the vision sensor are solved, and fast and accurate vision detection is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- EASY THINKING HANGZHOU TECH CO LTD
- Filing Date
- 2022-09-26
- Publication Date
- 2026-07-24
AI Technical Summary
Existing visual sensor calibration methods suffer from problems such as high equipment costs, unstable accuracy, the need for manual processing of feature points, large repeatability errors, and inability to perform online detection.
By employing a pre-calibration process, feature point coordinate information is collected through a visual sensor, the pose change matrix is calculated, and the initial transformation relationship is corrected, achieving fast and accurate matching between the visual sensor and the global coordinate system. This method is suitable for global positioning of both two-dimensional and three-dimensional sensors.
It improves the stability and accuracy of calibration results, avoids damage to the surface of the object being tested, enables online real-time detection, and saves calibration time and manpower.
Smart Images
Figure CN117764902B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of sensor positioning and calibration, and specifically to a detection method based on mobile devices and vision sensors. Background Technology
[0002] Currently, visual inspection technology has been widely applied in various fields of intelligent manufacturing, especially the flexible measurement method (visual sensor inspection method) that combines mobile devices (machines / guide rails) with visual sensors, which has attracted much attention due to its ability to provide more comprehensive visual measurement of the surface of the object being measured. This inspection method relies on a visual mathematical model to acquire information about the surface being measured. Before the visual sensor is officially applied, its relationship with the global coordinate system needs to be calibrated, that is, the visual sensor needs to be globally positioned / calibrated. Currently, sensor extrinsic calibration methods can be divided into passive positioning and active positioning.
[0003] Passive positioning utilizes mobile devices (guide rails, multi-axis robots) to move vision sensors to various measurement poses and measure the object being measured. It directly converts the rotation and translation parameters recorded by the mobile device itself during the movement into extrinsic parameters of the sensor. This method requires very high precision from the mobile device, resulting in high equipment purchase costs. Furthermore, due to the influence of temperature, component aging, etc., the rotation and translation parameters recorded by the mobile device itself have positioning errors, leading to unstable calibration results.
[0004] Active localization involves attaching feature points to the surface of the object being measured. A vision sensor then calculates extrinsic parameters based on the feature point images to determine the relationship between these points and the global coordinate system. This method offers high calibration accuracy, but it suffers from the following problems:
[0005] 1) Existing methods typically use adhesive / adhesive methods to fix feature points on the surface of the object being measured. After the object is measured, the feature points need to be manually removed to restore the surface morphology of the object. In special testing scenarios, the surface of the object cannot be touched, making it impossible to fix feature points. In such cases, existing active measurement methods cannot be implemented.
[0006] 2) The vision sensor needs to be moved to the measurement pose using a robot / guide rail before it can acquire an image. After the previous object is inspected, the new object needs to be reinstalled on the inspection station, and the vision sensor is moved to the measurement pose again to perform the measurement. During this process, due to the repeated positioning error of the mobile device (robot / guide rail), if the extrinsic parameters obtained from the measurement of the previous object are used directly, the sensor positioning will be inaccurate, affecting the detection results. Therefore, the existing method requires re-pasting feature points on the surface of the new object and using a photogrammetry system to obtain the standard coordinates of each feature point again, and then re-calibrating the process. The whole process is cumbersome and time-consuming, and is not suitable for online inspection processes. Summary of the Invention
[0007] To address the aforementioned technical problems, this invention provides a visual inspection method for flexible vision sensors. This method is fast, convenient, and accurate, and is applicable to various types of vision sensors (as long as a camera is available), such as two-dimensional sensors and three-dimensional sensors (structured light sensors, binocular vision sensors) for global positioning and visual inspection, and is especially suitable for online inspection processes.
[0008] The technical solution is as follows:
[0009] A detection method based on a mobile device and a vision sensor, wherein the vision sensor is installed on the mobile device for acquiring images of the object to be measured, characterized in that a pre-calibration process is first performed, during which the feature point coordinate information acquired by the vision sensor is recorded as calibration coordinates, and the transformation relationship between the sensor coordinate system and the global coordinate system at each measurement pose is recorded as the initial transformation relationship;
[0010] During testing, the transformation relationship between the visual sensor coordinate system and the global coordinate system was determined through the following steps:
[0011] Step 1: When the object to be measured reaches the preset position, the mobile device drives the vision sensor to the measurement pose according to the set trajectory;
[0012] Step 2: The vision sensor collects multiple feature points pre-set in the area surrounding the object to be tested. These feature points include existing geometric features in the detection station as well as manually added geometric features. At the same time, the vision sensor collects information about the area to be tested on the object to be tested, and obtains the coordinates of the detection points.
[0013] Using the acquired images, obtain the coordinate information of feature points and store them as measured coordinates;
[0014] The pose change matrix of the visual sensor is calculated by using the measured coordinates and the pre-stored calibration coordinates of the corresponding feature points.
[0015] Step 3: Use the pose change matrix to correct the pre-stored initial transformation relationship at the measurement pose to obtain the corrected transformation relationship, and record it as the final extrinsic parameter of the vision sensor at the measurement pose;
[0016] Step 4: Determine if there are any other measurement poses whose final extrinsic parameters have not been calculated. If so, the mobile device moves the vision sensor to another measurement pose and jumps to Step 2; if not, proceed directly to Step 5.
[0017] Step 5: Store the final extrinsic parameters of the vision sensor under each measurement pose; then use the final extrinsic parameters to transform the coordinates of the detection points to the global coordinate system; obtain the overall detection data of the object under test, and complete the visual inspection.
[0018] Furthermore, in step two, the solution method for the visual sensor pose change matrix is as follows:
[0019] When the vision sensor is a two-dimensional sensor, the measured coordinates and the calibration coordinates are both two-dimensional pixel coordinates. The pose change matrix of the vision sensor is solved by using the fundamental matrix estimation method or the homography matrix decomposition method.
[0020] When the vision sensor is a three-dimensional sensor, both the measured coordinates and the calibration coordinates are three-dimensional spatial coordinates. The pose change matrix of the vision sensor is solved using the SVD decomposition method.
[0021] Furthermore, in step three, the initial transformation relationship is corrected using the pose transformation matrix, resulting in the corrected transformation relationship:
[0022] R1 = RR0
[0023] T1 = RT0 + T
[0024] Where R and T represent the rotation and translation matrices in the pose transformation matrix, R0 and T0 represent the rotation and translation matrices in the initial transformation relationship, and R1 and T1 represent the rotation and translation matrices in the corrected transformation relationship.
[0025] Furthermore, the pre-calibration process is performed only once before the formal testing process, and includes the following steps:
[0026] S1. Select one test object from multiple test objects of the same model and record it as the reference test object;
[0027] A reference test object is installed in the testing station, and multiple feature points are set in the surrounding area and on the surface of the reference test object.
[0028] The coordinate information of each feature point in the global coordinate system is collected using standard measuring equipment and stored as standard coordinates.
[0029] S2. The mobile device drives the visual sensor to the measurement pose according to the set trajectory;
[0030] A visual sensor acquires images of multiple feature points;
[0031] The coordinate information of feature points is calculated using the acquired images and stored as calibration coordinates;
[0032] S3. Calculate the transformation relationship between the visual sensor coordinate system and the global coordinate system using standard coordinates and calibration coordinates, and store it as the initial transformation relationship at the current measurement pose.
[0033] S4. Determine if there are any other unresolved initial transformation relationships for the measured poses. If so, the mobile device moves the visual sensor to another measured pose and jumps to step S2. If not, proceed directly to step S5.
[0034] S5. Store the initial conversion relationship of the vision sensor under each measurement pose to complete the pre-calibration process.
[0035] Preferably, in step S2, the field of view of the vision sensor includes at least one feature point on the surface of the reference object and at least three feature points in the surrounding area of the reference object.
[0036] Preferably, in step S3, the method for solving the initial transformation relation R0T0 is as follows:
[0037] The objective function g(R0,T0) is established using the standard coordinates and measured coordinates of the feature points. Then, the transformation relationship corresponding to the minimum value of the objective function is solved iteratively using the optimization method, denoted as R0,T0.
[0038] When the vision sensor is a two-dimensional sensor, the objective function g(R0,T0) is:
[0039]
[0040] Where m represents the total number of feature points acquired by the camera, f(R0,T0,M,x) j ,y j ,z j ) represents the standard coordinates (x, y) of the j-th feature point using the camera pinhole imaging model. j ,y j ,z j Transform to the image plane, M represents the camera intrinsic parameter, (u j ,v j () represents the pixel coordinates of the j-th feature point in the image, i.e., the measured coordinates;
[0041] When the vision sensor is a binocular 3D sensor, the objective function g(R,T) is:
[0042]
[0043] Where, f1(R0,T0,M1,x) j ,y j ,z j ) represents the standard coordinates (x, y) of the j-th feature point using the pinhole imaging model of the left camera. j ,y j ,z j ) Transform to the left camera image plane, M1 represents the left camera intrinsic parameter, (u 1j ,v1j f2(R0,T0,R) represents the pixel coordinates of the j-th feature point in the left camera image, i.e., the measured coordinates; 12 ,T 12 ,M2,x j ,y j ,z j ) represents the standard coordinates (x, y) of the j-th feature point using the right camera pinhole imaging model. j ,y j ,z j Switch to the right camera image plane, R 12 ,T 12 These represent the rotation matrix and translation rectangle between the two cameras, respectively, and M2 represents the intrinsic parameters of the right camera. 2j ,v 2j ) represents the pixel coordinates of the j-th feature point in the right camera image, i.e., the measured coordinates.
[0044] Preferably, in S3, when the vision sensor is a three-dimensional sensor, the method for solving the initial transformation relationship is the SVD decomposition method;
[0045] When the vision sensor is a two-dimensional sensor, the method for solving the initial transformation relationship is the PnP method.
[0046] Preferably, in step S1, the standard measuring equipment includes a photogrammetry system, a laser tracker, and a coordinate measuring machine.
[0047] Preferably, the feature points include feature holes, standard spheres, and reflective / matte marking circles;
[0048] The standard coordinates, calibration coordinates, and measured coordinates are all the coordinates of the geometric center of the feature point.
[0049] Preferably, the feature points located in the area surrounding the reference test object are fixed on the ground around the test object, on the target, and on the surface of the support frame used to mount the test object;
[0050] The mobile device is a guide rail or a robot.
[0051] The visual sensor localization method in this approach is an active localization method. The localization information can be used for point cloud stitching and overall information acquisition of the measured object. The localization process has the following characteristics:
[0052] 1) The calibration process is not affected by the repeatability error of mobile devices (multi-axis robots, guide rails), and the calibration results are highly stable. There is no need to fix feature points on the surface of the object under test, and it will not cause any damage to the surface of the object.
[0053] 2) Before the formal detection begins, this method first performs a preliminary calibration process (steps S1 to S5) to obtain the initial transformation relationship RT. During this process, feature points are fixed and the calibration coordinates of the feature points are obtained using a vision sensor. The positions of the feature points set in the area around the object to be tested remain unchanged, and their corresponding calibration coordinates can be used as pre-stored data and directly used in subsequent detection steps (steps one to five), making the subsequent calibration process faster. This process can be performed before the sensor officially starts working, without occupying the production cycle, and only needs to be performed once. That is, only feature points need to be fixed on the surface of one object to be tested. During subsequent detection, feature points do not need to be fixed on the surfaces of other objects to be tested, saving manpower and detection time.
[0054] When the formal inspection begins (steps one through five), the objects to be tested are installed sequentially in the inspection station according to the production order. At this time, the vision sensor only needs to acquire feature point images set in the area around the object to be tested to quickly calculate the pose change matrix R0T0 generated by the vision sensor at the current measurement pose relative to the vision sensor in the pre-calibration process. By using R0T0 to correct the initial transformation relationship RT, the corrected transformation relationship R1T1 (the final extrinsic parameter of the current pose) can be obtained. This technical solution can save calibration time and avoid damage to the surface of the object to be tested while ensuring the positioning accuracy of the sensor; it can realize online real-time inspection.
[0055] In contrast, existing active positioning methods require attaching feature points and obtaining standard coordinates to the surface of the object each time a new object is replaced, which is time-consuming and cannot achieve online real-time detection.
[0056] 3) When solving the initial transformation relationship RT between the visual sensor coordinate system and the global coordinate system, in addition to the SVD decomposition method or PnP method mentioned in the prior art, the present invention can also use the optimization method to iteratively solve the problem and obtain a higher precision transformation matrix. Attached Figure Description
[0057] Figure 1 A schematic diagram showing the location of feature points in the pre-calibration process in a specific implementation embodiment;
[0058] Figure 2 This is a schematic diagram showing the location of the calibration point during testing in a specific implementation method. Detailed Implementation
[0059] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0060] A detection method based on mobile device and vision sensor. The vision sensor is installed on the mobile device (rail or robot) to acquire images of the object to be measured. A pre-calibration process is first performed. The feature point coordinate information acquired by the vision sensor during this process is recorded as the calibration coordinates. The transformation relationship between the sensor coordinate system and the global coordinate system at each measurement pose is recorded as the initial transformation relationship.
[0061] During testing, the transformation relationship between the visual sensor coordinate system and the global coordinate system was determined through the following steps:
[0062] Step 1: When the object to be measured reaches the preset position, the mobile device drives the vision sensor to the measurement pose according to the set trajectory;
[0063] Step 2: The vision sensor collects multiple feature points pre-set in the area surrounding the object to be measured. The feature points include the original geometric features in the detection station and the geometric features added manually.
[0064] At the same time, the vision sensor collects information about the area to be measured on the object and obtains the coordinates of the detection point;
[0065] Using the acquired images, obtain the coordinate information of feature points and store them as measured coordinates;
[0066] The pose change matrix of the visual sensor is calculated by using the measured coordinates and the pre-stored calibration coordinates of the corresponding feature points.
[0067] Step 3: Use the pose change matrix to correct the pre-stored initial transformation relationship at the measurement pose to obtain the corrected transformation relationship, and record it as the final extrinsic parameter of the vision sensor at the measurement pose;
[0068] Step 4: Determine if there are any other measurement poses whose final extrinsic parameters have not been calculated. If so, the mobile device moves the vision sensor to another measurement pose and jumps to Step 2; if not, proceed directly to Step 5.
[0069] Step 5: Store the final extrinsic parameters of the vision sensor under each measurement pose; then use the final extrinsic parameters to transform the coordinates of the detection points to the global coordinate system; obtain the overall detection data of the object under test, and complete the visual inspection.
[0070] Specifically, in step two, the solution method for the visual sensor pose change matrix is as follows:
[0071] When the vision sensor is a two-dimensional sensor, both the measured coordinates and the calibration coordinates are two-dimensional pixel coordinates. The pose change matrix of the vision sensor can be solved by using the fundamental matrix estimation method or the homography matrix decomposition method.
[0072] When the vision sensor is a three-dimensional sensor, both the measured coordinates and the calibration coordinates are three-dimensional spatial coordinates. The pose change matrix of the vision sensor is solved by using the SVD decomposition method.
[0073] In step three, the initial transformation relationship is corrected using the pose transformation matrix, resulting in the corrected transformation relationship:
[0074] R1 = RR0
[0075] T1 = RT0 + T
[0076] Where R and T represent the rotation and translation matrices in the pose transformation matrix, R0 and T0 represent the rotation and translation matrices in the initial transformation relationship, and R1 and T1 represent the rotation and translation matrices in the corrected transformation relationship.
[0077] More specifically, the pre-calibration process is performed only once before the formal testing process, and includes the following steps:
[0078] S1. Select one test object from multiple test objects of the same model and record it as the reference test object;
[0079] The reference test object is installed in the testing station, and multiple feature points are set in the surrounding area and on the surface of the reference test object;
[0080] Feature points include both existing geometric features in the inspection station and artificially added geometric features; such as feature holes, standard spheres, and reflective / matte marking circles.
[0081] Among them, the feature points located in the area surrounding the benchmark test object are fixed on the ground around the test object, on the target, and on the surface of the support frame used to mount the test object.
[0082] The coordinate information of each feature point in the global coordinate system is collected using standard measurement equipment (such as photogrammetry system, laser tracker and coordinate measuring machine) and stored as standard coordinates;
[0083] S2. The mobile device drives the visual sensor to the measurement pose according to the set trajectory;
[0084] A visual sensor acquires images of multiple feature points;
[0085] The coordinate information of feature points is calculated using the acquired images and stored as calibration coordinates;
[0086] S3. Calculate the transformation relationship between the visual sensor coordinate system and the global coordinate system using standard coordinates and calibration coordinates, and store it as the initial transformation relationship at the current measurement pose.
[0087] S4. Determine if there are any other unresolved initial transformation relationships for the measured poses. If so, the mobile device moves the visual sensor to another measured pose and jumps to step S2. If not, proceed directly to step S5.
[0088] S5. Store the initial conversion relationship of the vision sensor under each measurement pose to complete the pre-calibration process.
[0089] Among them, the standard coordinates, calibration coordinates, and measured coordinates are all the coordinates of the geometric center position of the feature point.
[0090] To complete the calculation, in step S2, the field of view of the vision sensor includes at least one feature point on the surface of the reference object and at least three feature points in the surrounding area of the reference object. In practice, to obtain better calculation accuracy, the field of view of the vision sensor includes 6 to 20 feature points on the surface of the reference object and 6 to 20 feature points in the surrounding area of the reference object.
[0091] Preferably, in step S3, the method for solving the initial transformation relation R0T0 includes the following:
[0092] Method 1: Establish the objective function g(R0,T0) using the standard coordinates and measured coordinates of the feature points, and then use the optimization method to iteratively solve the transformation relationship corresponding to the minimum value of the objective function, denoted as R0,T0;
[0093] When the vision sensor is a two-dimensional sensor, the objective function g(R0,T0) is:
[0094]
[0095] Where m represents the total number of feature points acquired by the camera, f(R0,T0,M,x) j ,y j ,z j ) represents the standard coordinates (x, y) of the j-th feature point using the camera pinhole imaging model. j ,y j ,z j Transform to the image plane, M represents the camera intrinsic parameter, (u j ,v j () represents the pixel coordinates of the j-th feature point in the image, i.e., the measured coordinates;
[0096] When the vision sensor is a binocular 3D sensor, the objective function g(R,T) is:
[0097]
[0098] Where, f1(R0,T0,M1,x) j ,yj ,z j ) represents the standard coordinates (x, y) of the j-th feature point using the pinhole imaging model of the left camera. j ,y j ,z j ) Transform to the left camera image plane, M1 represents the left camera intrinsic parameter, (u 1j ,v 1j f2(R0,T0,R) represents the pixel coordinates of the j-th feature point in the left camera image, i.e., the measured coordinates; 12 ,T 12 ,M2,x j ,y j ,z j ) represents the standard coordinates (x, y) of the j-th feature point using the right camera pinhole imaging model. j ,y j ,z j Switch to the right camera image plane, R 12 ,T 12 These represent the rotation matrix and translation rectangle between the two cameras, respectively, and M2 represents the intrinsic parameters of the right camera. 2j ,v 2j ) represents the pixel coordinates of the j-th feature point in the right camera image, i.e., the measured coordinates.
[0099] Method 2: When the vision sensor is a 3D sensor, the method for solving the initial transformation relationship is the SVD decomposition method;
[0100] Method 3: When the vision sensor is a two-dimensional sensor, the method for solving the initial transformation relationship is the PnP method.
[0101] Taking the visual inspection of the entire vehicle body-in-white as an example, this method will be illustrated by way of example. First, a pre-calibration process will be performed, such as... Figure 1 To determine the placement of feature points, multiple feature points are set in the surrounding area and on the surface of the reference body-in-white. The feature points include the original geometric features in the inspection station and the geometric features added manually.
[0102] In this embodiment, the feature points include: the punch holes in the support frame for mounting the body-in-white, the standard ball mounted on the support frame, and the reflective / matte marking circles pasted on the surface of the reference body-in-white and the surrounding ground, and on the target.
[0103] like Figure 2 As shown, the feature points in the area surrounding the reference test object include: the punch holes on the support frame itself, the standard ball mounted on the support frame, and the reflective / matte marking circles pasted on the surface of the reference body-in-white and the surrounding ground, and on the target;
[0104] Due to the large size of the vehicle body-in-white, a total of 8 measurement poses (numbered 1 to 8) were set up during the implementation. The robot (such as a 6-axis robot or a 7-axis robot) drove the vision sensor to the measurement pose in sequence according to the set trajectory.
[0105] After the pre-calibration process is completed, the following information will be obtained: 1) the initial transformation relationship of the vision sensor under each measurement pose; 2) the calibration coordinates of the feature points acquired by the vision sensor under each measurement pose.
[0106] During actual testing, the body-in-white to be tested is installed in the testing station. Only feature points within the area surrounding the baseline test object remain in the testing station, such as... Figure 2 As shown; Measurement pose (numbers 1-8);
[0107] The vision sensor uses these feature points to perform steps one through five; obtains sensor extrinsic parameters, and uses them to convert the detection point / point cloud information on the white body to the vehicle body coordinate system to evaluate the overall vehicle machining accuracy.
[0108] To verify the accuracy of the sensor extrinsic parameters calculated by the method of the present invention, the following accuracy verification experiment was also conducted in this embodiment. Seven feature points were randomly selected from the feature points in the area surrounding the reference object to be measured, and marked as numbers 1-7. The measured coordinates of the seven feature points acquired by the visual sensor were transformed to the global coordinate system using the corrected transformation relationship R1T1 to obtain the transformed coordinates. The difference between each transformed coordinate and the standard coordinates obtained by the photogrammetry system was calculated to obtain the coordinate deviation. The experimental data are shown in the table below:
[0109]
[0110] As shown in the table above, the coordinate deviation between the transformed coordinates obtained by the transformation relationship R1T1 in this method and the standard coordinates is less than 0.05mm, which meets the visual inspection requirements of high-precision manufacturing and processing fields (such as automotive parts inspection).
[0111] The foregoing description of specific exemplary embodiments of the present invention is for illustrative and descriptive purposes. It is not intended to be exhaustive, nor to limit the invention to the precise forms disclosed; obviously, many changes and variations are possible in accordance with the foregoing teachings. The exemplary embodiments were chosen and described to explain the specific principles of the invention and its practical application, thereby enabling others skilled in the art to implement and utilize various exemplary embodiments of the invention, as well as their different alternatives and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.
Claims
1. A detection method based on a mobile device and a vision sensor, wherein the vision sensor is installed on the mobile device for acquiring an image of the object to be measured, characterized in that, First, a pre-calibration process is performed. The feature point coordinates collected by the visual sensor during this process are recorded as calibration coordinates, and the transformation relationship between the sensor coordinate system and the global coordinate system at each measurement pose is recorded as the initial transformation relationship. During testing, the transformation relationship between the visual sensor coordinate system and the global coordinate system was determined through the following steps: Step 1: When the object to be measured reaches the preset position, the mobile device drives the vision sensor to the measurement pose according to the set trajectory; Step 2: The vision sensor collects multiple feature points pre-set in the area surrounding the object to be tested. These feature points include existing geometric features in the detection station as well as manually added geometric features. At the same time, the vision sensor collects information about the area to be tested on the object to be tested, and obtains the coordinates of the detection points. Using the acquired images, obtain the coordinate information of feature points and store them as measured coordinates; The pose change matrix of the visual sensor is calculated by using the measured coordinates and the pre-stored calibration coordinates of the corresponding feature points. Step 3: Use the pose change matrix to correct the pre-stored initial transformation relationship at the measurement pose to obtain the corrected transformation relationship, and record it as the final extrinsic parameter of the vision sensor at the measurement pose; Step 4: Determine if there are any other measurement poses whose final extrinsic parameters have not been calculated. If so, the mobile device moves the vision sensor to another measurement pose and jumps to Step 2; if not, proceed directly to Step 5. Step 5: Store the final extrinsic parameters of the vision sensor under each measurement pose; then use the final extrinsic parameters to transform the coordinates of the detection points to the global coordinate system; obtain the overall detection data of the object under test, and complete the visual inspection.
2. The detection method based on mobile devices and visual sensors as described in claim 1, characterized in that: In step two, the visual sensor pose change matrix is calculated as follows: When the vision sensor is a two-dimensional sensor, the measured coordinates and the calibration coordinates are both two-dimensional pixel coordinates. The pose change matrix of the vision sensor is solved by using the fundamental matrix estimation method or the homography matrix decomposition method. When the vision sensor is a three-dimensional sensor, both the measured coordinates and the calibration coordinates are three-dimensional spatial coordinates. The pose change matrix of the vision sensor is solved using the SVD decomposition method.
3. The detection method based on mobile devices and visual sensors as described in claim 1, characterized in that: In step three, the initial transformation relationship is corrected using the pose transformation matrix, resulting in the corrected transformation relationship: R1 = RR0 T1 = RT0 + T Where R and T represent the rotation and translation matrices in the pose transformation matrix, R0 and T0 represent the rotation and translation matrices in the initial transformation relationship, and R1 and T1 represent the rotation and translation matrices in the corrected transformation relationship.
4. The detection method based on mobile devices and visual sensors as described in claim 1, characterized in that: The pre-calibration process is performed only once before the formal testing process and includes the following steps: S1. Select one test object from multiple test objects of the same model and record it as the reference test object; A reference test object is installed in the testing station, and multiple feature points are set in the surrounding area and on the surface of the reference test object. The coordinate information of each feature point in the global coordinate system is collected using standard measuring equipment and stored as standard coordinates. S2. The mobile device drives the visual sensor to the measurement pose according to the set trajectory; A visual sensor acquires images of multiple feature points; The coordinate information of feature points is calculated using the acquired images and stored as calibration coordinates; S3. Calculate the transformation relationship between the visual sensor coordinate system and the global coordinate system using standard coordinates and calibration coordinates, and store it as the initial transformation relationship at the current measurement pose. S4. Determine if there are any other unresolved initial transformation relationships for the measured poses. If so, the mobile device moves the visual sensor to another measured pose and jumps to step S2. If not, proceed directly to step S5. S5. Store the initial conversion relationship of the vision sensor under each measurement pose to complete the pre-calibration process.
5. The detection method based on mobile devices and visual sensors as described in claim 4, characterized in that: In step S2, the field of view of the vision sensor includes at least one feature point on the surface of the reference object and at least three feature points in the surrounding area of the reference object.
6. The detection method based on mobile devices and visual sensors as described in claim 4, characterized in that: In step S3, the initial transformation relation is solved. The method is as follows: Establish the objective function using the standard coordinates and measured coordinates of the feature points. Then, the transformation relationship corresponding to the minimum value of the objective function is solved iteratively using the optimization method, denoted as . ; Wherein, when the vision sensor is a two-dimensional sensor, the objective function for: Where m represents the total number of feature points captured by the camera. This indicates that the standard coordinates of the j-th feature point are determined using a camera pinhole imaging model. Transform to the image plane, M represents the camera intrinsic parameter, ( , () represents the pixel coordinates of the j-th feature point in the image, i.e., the measured coordinates; When the vision sensor is a binocular 3D sensor, the objective function for: in, This indicates that the standard coordinates of the j-th feature point are obtained using the pinhole imaging model of the left camera. Switch to the left camera image plane. Indicates the intrinsic parameters of the left camera, ( , () represents the pixel coordinates of the j-th feature point in the left camera image; This indicates that the standard coordinates of the j-th feature point are obtained using the right camera pinhole imaging model. Switch to the right camera image plane. These represent the rotation matrix and translation rectangle between the two cameras, respectively. Indicates the intrinsic parameters of the right camera, ( , () represents the pixel coordinates of the j-th feature point in the right camera image.
7. The detection method based on mobile devices and visual sensors as described in claim 4, characterized in that: In S3, when the vision sensor is a three-dimensional sensor, the method for solving the initial transformation relationship is the SVD decomposition method; When the vision sensor is a two-dimensional sensor, the method for solving the initial transformation relationship is the PnP method.
8. The detection method based on mobile devices and visual sensors as described in claim 4, characterized in that: In step S1, the standard measuring equipment includes a photogrammetry system, a laser tracker, and a coordinate measuring machine.
9. The detection method based on mobile devices and visual sensors as described in claim 1 or 4, characterized in that: The feature points include feature holes, standard spheres, and reflective / matte marking circles; The standard coordinates, calibration coordinates, and measured coordinates are all the coordinates of the geometric center of the feature point.
10. The detection method based on mobile devices and visual sensors as described in claim 1 or 4, characterized in that: Feature points located within the area surrounding the reference test object are fixed on the ground surrounding the test object, on the target, and on the surface of the support frame used to mount the test object. The mobile device is a guide rail or a robot.