A method for real-time obtaining of external camera parameters for a moving object to be measured

Abstract

The present invention discloses a method for real-time acquisition of the external parameters of a camera for a moving object, including: when the object to be measured triggers the sensor, the roller bed stops, and the pulse number recorded by the encoder is denoted as A0, and the rotation and translation relationship R0T0 is obtained; the roller bed advances, and each time the camera captures an image, the pulse number recorded by the encoder is extracted and the rotation and translation relationship R i T i between the coordinate system of the object to be measured and the camera coordinate system at this moment is calculated, and the calculated R i T i is stored as the real-time external parameters of the camera. In this method, the translation distance D of the object to be measured between two adjacent pulse numbers is first calibrated, and then the real-time moving distance value of the object to be measured is accurately obtained. Based on the distance value, the change amount of the translation matrix is solved to obtain the real-time external parameters, aiming to improve the acquisition accuracy of the real-time external parameters of the camera, ensure the three-dimensional positioning accuracy during subsequent image detection, and control the three-dimensional positioning error within 3 mm.

Description

Technical Field

[0001] The present invention relates to the field of external parameter calibration, and particularly to a method for real-time obtaining of camera external parameters for a moving object to be measured. Background Art

[0002] In industrial inspection scenarios, for large objects to be measured, the positions to be measured are usually distributed in different regions of the object to be measured. The field of view of a single camera cannot cover all the regions to be measured. Therefore, in order to enable the camera to collect multiple regions on the object to be measured, the following two methods can be adopted:

[0003] Method 1: Increase the number of cameras. However, this method adds additional equipment costs.

[0004] Method 2: Adopt the method of relative movement between the object to be measured and the camera. In this method, a batch of objects to be measured are sequentially placed on a moving device (roller bed) and perform continuous translational movement relative to the camera; for each object to be measured, the camera collects images of the object to be measured during the movement multiple times at a preset frame rate. Among them, only a partial region of the object to be measured is included in a single image. As the object to be measured translates, the multiple collected images can cover the entire region of the object to be measured from beginning to end. This method does not require additional equipment costs and has received more attention. When using this method, in order to achieve global stitching of images or perform three-dimensional calculation on feature points in the images, it is necessary to pre-calibrate the relative position relationship (camera external parameters) between the coordinate system of the moving object to be measured and the camera coordinate system. The existing external parameter calibration methods generally perform external parameter calibration for objects to be measured in a static state, and there are few mentions of external parameter calibration methods for moving objects. Summary of the Invention

[0005] In order to solve the above technical problems, the present invention provides a method for real-time obtaining of camera external parameters for a moving object to be measured, aiming to improve the accuracy of obtaining real-time camera external parameters and ensure the three-dimensional positioning accuracy during subsequent image detection.

[0006] Therefore, the technical solution of the present invention is as follows:

[0007] A method for real-time obtaining of camera external parameters for a moving object to be measured, wherein the object to be measured is placed on a roller bed, the camera is fixed around the roller bed, and sensors are fixed around the camera, and the sensors are used to sense whether the object to be measured is in place;

[0008] The roller bed is provided with rollers, the rollers are controlled by a motor, the motor is provided with an encoder. When the motor drives the rollers to rotate at a constant speed, the encoder continuously records the pulse number, and the roller bed moves forward; when the motor controls the rollers not to rotate, the encoder stops counting, and the roller bed stops;

[0009] During detection, the roller bed moves forward, and a batch of objects to be measured are successively placed on the roller bed and perform continuous translational motion relative to the camera. When a single object to be measured triggers the sensor, the camera starts to collect images of the object to be measured during the motion multiple times at a preset frame rate. Among them, only a partial area of the object to be measured is included in a single image. As the object to be measured translates, multiple collected images can cover the entire area of the object to be measured from beginning to end.

[0010] Before the formal detection starts, one of the objects to be measured is used to perform the following steps to obtain the relative position relationship between the coordinate system of the object to be measured and the camera coordinate system at different times. The X-axis of the coordinate system of the object to be measured is parallel to the translation direction of the roller bed, and the Y-axis is perpendicular to the translation direction:

[0011] 1) When the object to be measured triggers the sensor, the roller bed stops, and the pulse number recorded by the encoder at this time is extracted and denoted as; this position is recorded as the initial position and camera calibration is performed to obtain the rotation and translation relationship R0T0 between the coordinate system of the object to be measured and the camera coordinate system. Among them, R0 represents the rotation matrix, and T0 represents the translation matrix, which includes the translation components T x0 、T y0 、T z0 ;

[0012] 2) The roller bed moves forward, and the encoder in the roller bed continuously records the pulse number;

[0013] Each time the camera takes a picture, the pulse number recorded by the encoder is extracted and the rotation and translation relationship R i T i between the coordinate system of the object to be measured and the camera coordinate system at this moment is calculated using the following formula:

[0014] Rotation matrix R i =R0, translation matrix ;

[0015] Among them, i represents the i-th image, S = D×(A i -A0); A i is the pulse number recorded by the encoder when the i-th image is collected, D is the distance of the object to be measured translated between two adjacent pulse numbers calibrated in advance, and θ is the angle between the forward direction of the object to be measured and the forward direction of the roller bed calibrated in advance;

[0016] The calculated R i T i is stored as the real-time external parameters of the camera.

[0017] Furthermore, the pre-calibration process of D and θ is as follows:

[0018] When the object to be measured triggers the sensor, the laser tracker obtains the three-dimensional coordinates of the target ball pre-fixed on the object to be measured;

[0019] Let the roller bed move forward and stop after a preset time. At this time, the laser tracker acquires the three-dimensional coordinates of the marked points on the object to be measured again, and records the pulse number recorded by the encoder at this time as A';

[0020] Denote the angle between the line connecting the two acquired three-dimensional coordinates and the X-axis as θ;

[0021] Calculate the spatial distance between the two three-dimensional coordinates, denoted as L;

[0022] Calculate: , where L is the spatial distance between the two acquired three-dimensional coordinates.

[0023] For the case where the object to be measured slips on the roller bed, the present invention also discloses a method for real-time obtaining of the external parameters of a camera for a moving object to be measured. The object to be measured is placed on the roller bed, the camera is fixed around the roller bed, and a sensor is fixed around the camera. The sensor is used to sense whether the object to be measured is in place;

[0024] The roller bed is provided with rollers, the rollers are controlled by a motor, the motor is provided with an encoder. When the motor drives the rollers to rotate uniformly, the encoder continuously records the pulse number and the roller bed moves forward; when the motor controls the rollers not to rotate, the encoder stops counting and the roller bed stops;

[0025] During detection, the roller bed moves forward, and a batch of objects to be measured are sequentially placed on the roller bed and perform continuous translational motion relative to the camera; when a single object to be measured triggers the sensor, the camera starts to collect images of the object to be measured during the motion multiple times at a preset frame rate. Among them, only a partial area of the object to be measured is included in a single image. As the object to be measured translates, the multiple collected images can cover the entire area of the object to be measured from beginning to end;

[0026] Before the formal detection starts, use one of the objects to be measured to perform the following steps to obtain the relative position relationship between the coordinate system of the object to be measured and the camera coordinate system at different times. The X-axis of the coordinate system of the object to be measured is parallel to the translation direction of the roller bed, and the Y-axis is perpendicular to the translation direction:

[0027] S1. When the object to be measured triggers the sensor, the roller bed stops, and extract the pulse number recorded by the encoder at this time and denote it as A0; denote this position as the initial position and perform camera calibration to obtain the rotation and translation relationship R0T0 between the coordinate system of the object to be measured and the camera coordinate system; where, R0 represents the rotation matrix, and T0 represents the translation matrix, which includes the translation components T x0 、T y0 、T z0 ;

[0028] S2. The roller bed moves forward, and the encoder in the roller bed continuously records the pulse sequence number. The camera acquires images at a preset frame rate. During the forward movement of the roller bed, the roller bed is randomly stopped multiple times, and step B is performed each time it stops until the object to be measured is translated out of the detection area, then step S3 is executed;

[0029] The said step B is as follows:

[0030] The camera is turned off, and the pulse sequence number currently recorded by the encoder is denoted as A'. j , and the distance L by which the current position of the object to be measured is translated compared to the initial position is obtained j , and the angle θ between the forward direction of the object to be measured and the forward direction of the roller bed j ; j represents the number of times step B is executed, j = 1, 2, 3... M, and M represents the total number of times;

[0031] Calculate ;

[0032] Calculate the rotation and translation relationship R'' j between the coordinate system of the object to be measured and the camera coordinate system during the period when the camera acquires images with the pulse sequence numbers A0 to A' i T'' i :

[0033] The rotation matrix R'' i = R0, the translation matrix ;

[0034] where i represents the i-th image, S'' = D j ×(A i - A0);

[0035] A i represents the pulse sequence number recorded by the encoder when the i-th image is acquired;

[0036] Use R'' i T'' i to solve the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates;

[0037] Calculate the deviation value between the measured coordinates and the coordinates of the feature points in the digital model;

[0038] The roller bed moves forward, and the encoder in the roller bed continuously records the pulse sequence number. The camera is turned on and continues to acquire images at the preset frame rate;

[0039] S3. From the multiple deviation values calculated in step S2, select the D j and θ j corresponding to the minimum deviation value, and denote them as D and θ respectively;

[0040] Calculate the rotation and translation relationship R between the coordinate system of the object to be measured and the camera coordinate system each time the camera captures an image i T i :

[0041] Rotation matrix R i = R0, translation matrix ;

[0042] where i represents the i-th image, S = D × (A i - A0);

[0043] Store the calculated R i T i as the real-time external parameters of the camera

[0044] Furthermore, in step B, obtain the angle θ between the forward direction of the object to be measured and the forward direction of the roller bed j , the method is as follows:

[0045] In step S1, use a laser tracker to obtain the three-dimensional coordinates Q of the target ball pre-fixed on the object to be measured

[0046] In step B, use a laser tracker to obtain the three-dimensional coordinates Q of the target ball on the object to be measured again j , and record the angle between the line connecting the three-dimensional coordinates Q j and the three-dimensional coordinates Q and the X-axis as θ j .

[0047] Furthermore, the methods for the distance by which the current position of the object to be measured is translated compared to the initial position include the following two types:

[0048] Method 1:

[0049] Record the spatial distance between the three-dimensional coordinates Q j and the three-dimensional coordinates Q as L j ;

[0050] Method 2:

[0051] In step S1, the camera captures an image of the surface of the object to be measured, extracts the pixel coordinates (u, v) of the surface feature points of the object to be measured, and searches for the three-dimensional coordinates (X1, Y1, Z1) of this feature point on the digital model of the object to be measured

[0052] In step B, in the latest image captured by the camera, extract the pixel coordinates (u', v') of another feature point, and search for the three-dimensional coordinates (X1', Y1', Z1') of this feature point on the digital model of the object to be measured

[0053] , s represents the pixel equivalent

[0054] Preferably, in step S2, during the forward movement of the roller bed, the roller bed is stopped at equal intervals for multiple times.

[0055] Preferably, the method further includes the following steps:

[0056] I. In the interval [D - δ, D + δ], take values multiple times according to a preset step size. After each value is taken, the following processing is performed:

[0057] Record the currently obtained value as D';

[0058] Calculate the translation matrix using D'; ;

[0059] where S' = D'×(A i - A0);

[0060] Calculate the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured using R i T i ', and record them as the measured coordinates;

[0061] Calculate the deviation value between the measured coordinates and the coordinates of the feature points in the digital model;

[0062] II. Select the value of D' corresponding to the minimum deviation value, and record the R i T i ' obtained by solving this value as the final R i T i .

[0063] Furthermore, the method for calculating the measured coordinates is as follows:

[0064] Construct a space straight line L using the pixel coordinates of the camera optical center and the feature points in the image;

[0065] Convert the coordinate system of the object digital model to the camera coordinate system using R i T i '. The space straight line L intersects with the converted object digital model, and record the intersection coordinates as the three-dimensional coordinates of the feature points in the camera coordinate system;

[0066] Then convert the three-dimensional coordinates to the coordinate system of the object to be measured using R i T i ' to obtain the measured coordinates.

[0067] Furthermore, in step I, there are multiple feature points distributed in different images. For each feature point, first find the R i T i ' corresponding to the image where it is located, and then use the found R i T i ' to calculate the measured coordinates and deviation values;

[0068] Take the average of the deviation values corresponding to each feature point as the final deviation value.

[0069] Preferably, δ ranges from 0.3 mm to 1 mm, and the preset step size ranges from 0.1δ to 0.5δ.

[0070] This method has the following characteristics:

[0071] 1. This method first calibrates the distance D of the object to be measured translated between two adjacent pulse numbers, and then accurately obtains the real-time moving distance value of the object to be measured. Based on the distance value, the change amount of the translation matrix is calculated to obtain the real-time external parameters.

[0072] 2. Iterate the value of D within the floating range, select the best value for three-dimensional positioning, further eliminate the errors introduced in the calculation process, control the three-dimensional positioning error within 3 mm, and can be applied to high-precision application scenarios such as precise three-dimensional positioning of defects and automatic grinding of defects.

[0073] 3. Let the roller bed stop multiple times in the detection area, and calibrate to obtain multiple values of D j , further solve the problem of inaccurate calibration of the value of D due to slipping during the vehicle operation, and improve the accuracy and stability of the calibration results. Brief Description of the Drawings

[0074] Figure 1 It is a schematic diagram of the positional relationship between the camera and the object to be measured. Detailed Embodiments

[0075] The technical solutions of the present invention will be described in detail below with reference to the drawings and embodiments.

[0076] Embodiment 1

[0077] A method for real-time obtaining of the external parameters of a camera for a moving object to be measured, the object to be measured is placed on a roller bed, the camera is fixed around the roller bed, and a sensor (such as a photoelectric switch) is fixed around the camera, and the sensor is used to sense whether the object to be measured is in place;

[0078] The roller bed is provided with rollers, the rollers are controlled by a motor, the motor is provided with an encoder, when the motor drives the rollers to rotate uniformly, the encoder continuously records the pulse numbers, and the roller bed moves forward; when the motor controls the rollers not to rotate, the encoder stops counting and the roller bed stops;

[0079] During detection, the roller bed moves forward, and a batch of objects to be measured are sequentially placed on the roller bed and perform continuous translational motion relative to the camera; when a single object to be measured triggers the sensor, the camera starts to collect images of the object to be measured during the movement multiple times at a preset frame rate. Among them, only a partial area of the object to be measured is included in a single image, and as the object to be measured translates, the multiple collected images can cover the entire area of the object to be measured from beginning to end;

[0080] During use, retrieve the external camera parameters R corresponding to the i-th pre-calibrated image. i T i Stitch the images / perform 3D calculation.

[0081] As Figure 1 shown, taking the vehicle body to be measured as an example, the front of the vehicle, the body, and the rear of the vehicle pass through the camera in sequence, and the collected images include a front-of-vehicle image, a body image, and a rear-of-vehicle image in sequence.

[0082] Before the formal start of the detection, use one of the objects to be measured to perform the following steps to obtain the relative position relationship (external camera parameters) between the coordinate system of the object to be measured and the camera coordinate system at different times. The X-axis of the coordinate system of the object to be measured is parallel to the translation direction of the roller bed, and the Y-axis is perpendicular to the translation direction.

[0083] 1) When the object to be measured triggers the sensor, the roller bed stops, and the pulse number recorded by the encoder at this time is extracted and denoted as; this position is recorded as the initial position and camera calibration is performed to obtain the rotation and translation relationship R0T0 between the coordinate system of the object to be measured and the camera coordinate system; where R0 represents the rotation matrix and T0 represents the translation matrix, which contains translation components T x0 、T y0 、T z0 ;

[0084] This step is carried out by using a conventional method, such as calibration with the aid of a laser tracker.

[0085] 2) The roller bed advances, and the encoder in the roller bed continuously records the pulse numbers.

[0086] Each time the camera captures an image, the pulse number recorded by the encoder is extracted and the rotation and translation relationship R i T i between the coordinate system of the object to be measured and the camera coordinate system at this moment is calculated by using the following formula:

[0087] Rotation matrix R i = R0, translation matrix ;

[0088] where i represents the i-th image, S = D×(A i - A0); A i is the pulse number recorded by the encoder when the i-th image is captured, is the pre-calibrated translation distance of the object to be measured between two adjacent pulse numbers, and θ is the pre-calibrated angle between the advancing direction of the object to be measured and the advancing direction of the roller bed;

[0089] The calculated R i T i is stored as the real-time external camera parameters.

[0090] In this embodiment, the pre-calibration process of D and θ is as follows:

[0091] As Figure 1 shown, when the object to be measured triggers the sensor (position A), the laser tracker acquires the three-dimensional coordinates of the target ball pre-fixed on the object to be measured;

[0092] Let the roller bed move forward and stop after a preset time (such as 10 s, 25 s) (position B). At this time, the laser tracker acquires the three-dimensional coordinates of the marked point on the object to be measured again and records the pulse number recorded by the encoder at this time as A';

[0093] Record the angle between the line connecting the two acquired three-dimensional coordinates and the X-axis of the coordinate system of the object to be measured as θ;

[0094] Calculate the spatial distance between the two three-dimensional coordinates, denoted as L;

[0095] Calculate: , where L is the spatial distance between the two acquired three-dimensional coordinates. The object to be measured is placed parallel on the roller bed (the included angle value is less than 10°). Due to its large size and weight, the included angle value usually does not change during translation.

[0096] To obtain a more accurate R i T i , perform step-by-step iteration on D, select the value that satisfies the best three-dimensional positioning (the minimum positioning error) to improve the accuracy of the external parameter solution. For this purpose, the preferred implementation further includes step 3):

[0097] I. In the interval [D - δ, D + δ], take multiple values according to the preset step size. After each value is taken, perform the following processing:

[0098] Record the currently obtained value as D';

[0099] Calculate the translation matrix using D' ;

[0100] where S' = D' × (A i - A0);

[0101] Calculate the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured using R i T i ', denoted as the measured coordinates;

[0102] Calculate the deviation value between the measured coordinates and the coordinates of the feature points in the digital model;

[0103] II. Select the value of D' corresponding to the minimum deviation value, and record the R i T i ' obtained by solving this value as the final R i Ti 。

[0104] Among them, δ takes values in the range of 0.3 mm to 1 mm, and the preset step size takes values in the range of 0.1δ to 0.5δ.

[0105] More specifically, the method for calculating the measured coordinates is as follows:

[0106] Construct a space straight line L by using the pixel coordinates of the camera optical center and the feature points in the image;

[0107] Use R i T i ' to convert the digital model coordinate system of the object to be measured into the camera coordinate system. The space straight line L intersects with the digital model of the object to be measured after conversion, and record the intersection coordinates as the three-dimensional coordinates of the feature points corresponding in the camera coordinate system;

[0108] Then use R i T i ' to convert the three-dimensional coordinates into the coordinate system of the object to be measured to obtain the measured coordinates.

[0109] Among them, in order to increase the binding force, select the most reasonable value D'. In this embodiment:

[0110] There are multiple feature points distributed in different images. For each feature point, first find the corresponding R i T i ', and then use the found R i T i ' to calculate the measured coordinates and deviation values;

[0111] Take the average of the deviation values corresponding to each feature point as the final deviation value;

[0112] For example: D = 3.5 mm. On the object to be measured, select 3 feature points from front to back. These 3 feature points are distributed in the 1st, 3rd, and 4th images.

[0113] I. In the interval of [3.5 - 0.5, 3.5 + 0.5], take multiple values according to the preset step size of 0.2 mm. Then the first value is 3, the second value is 3.2, the third value is 3.4...

[0114] After each value is taken, perform the following processing:

[0115] Record the currently obtained value as D';

[0116] Calculate the translation matrix using D' ;

[0117] Among them, S' = D' × (A i - A0);

[0118] Use Ri T i Solve the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates;

[0119] On the object to be measured, select 3 feature points from front to back. These 3 feature points are distributed in the 1st, 3rd, and 4th images. Then for feature point 1:

[0120] Use R1T1' to convert the digital-analog coordinate system of the object to be measured to the camera coordinate system. The space line L intersects with the converted digital-analog of the object to be measured, and record the intersection coordinates as the three-dimensional coordinates of feature point 1 in the camera coordinate system;

[0121] Then use R i T i ' to convert the three-dimensional coordinates to the coordinate system of the object to be measured, and obtain the measured coordinate 1.

[0122] Calculate the deviation value 1 between the measured coordinate 1 and the coordinates of feature point 1 in the digital model;

[0123] For feature point 2:

[0124] Use R3T3' to convert the digital-analog coordinate system of the object to be measured to the camera coordinate system. The space line L intersects with the converted digital-analog of the object to be measured, and record the intersection coordinates as the three-dimensional coordinates of feature point 1 in the camera coordinate system;

[0125] Then use R3T3' to convert the three-dimensional coordinates to the coordinate system of the object to be measured, and obtain the measured coordinate 2;

[0126] Calculate the deviation value 2 between the measured coordinate 2 and the coordinates of feature point 2 in the digital model;

[0127] For feature point 3:

[0128] Use R1T1' to convert the digital-analog coordinate system of the object to be measured to the camera coordinate system. The space line L intersects with the converted digital-analog of the object to be measured, and record the intersection coordinates as the three-dimensional coordinates of feature point 1 in the camera coordinate system;

[0129] Then use R i T i ' to convert the three-dimensional coordinates to the coordinate system of the object to be measured, and obtain the measured coordinate 3;

[0130] Calculate the deviation value between the measured coordinates and the coordinates of the feature points in the digital model;

[0131] Calculate the deviation value 3 between the measured coordinate 3 and the coordinates of feature point 3 in the digital model;

[0132] Take the average of the deviation values 1, 2, and 3 as the final deviation value;

[0133] II. Select the value of D' corresponding to the minimum deviation value, and calculate the R obtained from this value i T i ' and denote it as the final R i T i .

[0134] In this embodiment, for the case where the object to be measured does not slip on the roller bed, D only needs to be calibrated once to calculate R i T i , and at the same time, in order to eliminate the errors introduced during the calculation, D is iterated step by step, and the value that satisfies the best three-dimensional positioning is selected to improve the accuracy of the external parameter calculation and control the three-dimensional positioning error within 3 mm.

[0135] Embodiment 2

[0136] A method for real-time obtaining of the external parameters of a camera for a moving object to be measured, where the object to be measured is placed on a roller bed, the camera is fixed around the roller bed, and sensors are fixed around the camera, and the sensors are used to sense whether the object to be measured is in place;

[0137] The roller bed is provided with rollers, the rollers are controlled by a motor, the motor is provided with an encoder, when the motor drives the rollers to rotate uniformly, the encoder continuously records the pulse number, and the roller bed advances; when the motor controls the rollers not to rotate, the encoder stops counting and the roller bed stops;

[0138] During detection, the roller bed advances, and a batch of objects to be measured are sequentially placed on the roller bed and perform continuous translational motion relative to the camera; when a single object to be measured triggers the sensor, the camera starts to collect images of the object to be measured during the motion multiple times at a preset frame rate. Among them, only a partial area of the object to be measured is included in a single image, and as the object to be measured translates, the multiple images collected can cover the entire area of the object to be measured from beginning to end;

[0139] Before the formal detection starts, use one of the objects to be measured to perform the following steps to obtain the relative position relationship between the coordinate system of the object to be measured and the camera coordinate system at different times. The X-axis of the coordinate system of the object to be measured is parallel to the translation direction of the roller bed, and the Y-axis is perpendicular to the translation direction:

[0140] S1. When the object to be measured triggers the sensor (such as a photoelectric switch), the roller bed stops, and the pulse number recorded by the encoder at this time is extracted and denoted as; record this position as the initial position and perform camera calibration to obtain the rotation and translation relationship R0T0 between the coordinate system of the object to be measured and the camera coordinate system; where R0 represents the rotation matrix, and T0 represents the translation matrix, which includes the translation components T x0 , T y0 , T z0 ;

[0141] Specifically, when implementing, use a laser tracker to obtain the three-dimensional coordinates Q of the target ball pre-fixed on the object to be measured

[0142] S2. The roller bed advances, and the encoder in the roller bed continuously records the pulse sequence numbers. The camera acquires images at a preset frame rate. During the advancement of the roller bed, the roller bed is made to stop randomly multiple times, and step B is performed each time it stops until the object to be measured on the roller bed is translated out of the detection area (for example: two sensors are set up, the first one is used to sense the entry of the object to be measured into the detection area, and the second one is used to sense the exit of the object to be measured from the detection area. When the object to be measured triggers the second sensor, it means that the tail of the object to be measured has been imaged by the camera and has been translated out of the detection area), then step S3 is executed;

[0143] Specifically in implementation, the difference between the pulse sequence numbers recorded by the encoder between two adjacent stop positions can be made to be within the range of 3000 to 150000. Preferably, the stop positions of the roller bed are evenly distributed in the trajectory of the roller bed advancing within the detection area. That is: during the advancement of the roller bed, the roller bed is made to stop at equal intervals multiple times; in implementation, multiple sensors can be evenly set in the entire detection area, and each time the object to be measured triggers a sensor, the roller bed stops.

[0144] Among them, step B is:

[0145] The camera is turned off (the roller bed stops), the pulse sequence number currently recorded by the encoder is denoted as, the translation distance of the current position of the object to be measured compared to the initial position is obtained, and the included angle θ between the advancing direction of the object to be measured and the advancing direction of the roller bed is obtained j ; j represents the number of times step B is executed, j = 1, 2, 3... M, and M represents the total number of times;

[0146] L j 、θ j The specific method for obtaining and θ is: use a laser tracker to obtain the three-dimensional coordinates Q of the target ball on the object to be measured again j , and denote the included angle between the line connecting the three-dimensional coordinates Q j and the three-dimensional coordinates Q and the X-axis as θ j ; j represents the number of times step B is executed, j = 1, 2, 3... M, and M represents the total number of times; denote the spatial distance between the three-dimensional coordinates Q j and the three-dimensional coordinates Q as L j ;

[0147] Specifically, for the included angle θ j , the method is as follows:

[0148] In step S1, use a laser tracker to obtain the three-dimensional coordinates Q of the target ball pre-fixed on the object to be measured;

[0149] In step B, use a laser tracker to obtain the three-dimensional coordinates Q of the target ball on the object to be measured again j , and the three-dimensional coordinates Q jThe included angle between the line connecting to the three-dimensional coordinate Q and the X-axis of the coordinate system of the object to be measured is denoted as θ j 。

[0150] Distance L j can be obtained in the following two ways:

[0151] Method 1:

[0152] Denote the spatial distance between the three-dimensional coordinate Q j and the three-dimensional coordinate Q as L j ;

[0153] Method 2:

[0154] In step S1, the camera captures the surface image of the object to be measured, extracts the pixel coordinates (u, v) of the surface feature points of the object to be measured therein, and searches for the three-dimensional coordinates (X1, Y1, Z1) of this feature point on the digital model of the object to be measured;

[0155] In step B, in the image newly captured by the camera, extract the pixel coordinates (u', v') of another feature point therein, and search for the three-dimensional coordinates (X1', Y1', Z1') of this feature point on the digital model of the object to be measured;

[0156] , where s represents the pixel equivalent.

[0157] Calculate ;

[0158] Calculate the rotation and translation relationship R'' j between the coordinate system of the object to be measured and the camera coordinate system during the camera image acquisition period from pulse sequence number A0 to A' i T'' i :

[0159] Rotation matrix R'' i = R0, translation matrix ;

[0160] where i represents the i-th image, S'' = D j ×(A i - A0);

[0161] A i represents the pulse sequence number recorded by the encoder when the i-th image is acquired;

[0162] Use R'' i T'' i to solve the coordinates of the feature point in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates;

[0163] Calculate the deviation value between the measured coordinates and the coordinates of the feature point in the digital model;

[0164] The roller bed moves forward, the encoder in the roller bed continuously records the pulse number, the camera is turned on, and continues to collect images at the preset frame rate;

[0165] For increasing the binding force, it is preferably implemented as:

[0166] There are multiple feature points distributed in different images. For each feature point, first find the corresponding R'' i T'' i of the image where it is located, and then use the found R'' i T'' i to calculate the measured coordinates and deviation values;

[0167] Take the average of the deviation values corresponding to each feature point as the final deviation value.

[0168] S3. From the multiple deviation values calculated in step S2, select the D j and θ j corresponding to the minimum deviation value, and denote them as D and θ respectively;

[0169] Use them to calculate the rotation and translation relationship R i T i between the coordinate system of the object to be measured and the camera coordinate system each time the camera captures an image:

[0170] The rotation matrix R i = R0, the translation matrix ;

[0171] where i represents the i-th image, S = D × (A i - A0);

[0172] Store the calculated R i T i as the real-time external parameters of the camera.

[0173] For example: Exemplarily, in step S2, the roller bed moves forward, the encoder in the roller bed continuously records the pulse number, and the camera captures images at the preset frame rate; during the forward movement of the roller bed, let the roller bed stop twice, and perform step B each time it stops:

[0174] Until the object to be measured on the roller bed is translated out of the detection area, execute step S3;

[0175] When it stops for the first time, perform step B:

[0176] The camera is turned off, record the current pulse number recorded by the encoder as A’1 = 2000, obtain the distance by which the current position of the object to be measured is translated compared to the initial position, and the angle θ1 between the forward direction of the object to be measured and the forward direction of the roller bed;

[0177] Calculate ;

[0178] Calculate the rotation and translation relationship R'' between the coordinate system of the object to be measured and the camera coordinate system during the camera image acquisition period from pulse sequence number A0 to A’1 i T'' i :

[0179] Rotation matrix R'' i = R0, translation matrix ;

[0180] where i represents the i-th image, and S'' = D1 × (A i - A0);

[0181] A i represents the pulse sequence number recorded by the encoder when the i-th image is acquired;

[0182] Use R'' i T'' i to solve the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates;

[0183] Calculate the deviation value between the measured coordinates and the coordinates of the feature points in the digital model;

[0184] For example, when stopping this time, a total of 6 images are acquired, and there are two feature points, which are distributed in the 3rd and 5th images. Then use R''3T''3 and R''5T''5 to solve the measured coordinates and deviation values;

[0185] Take the average value of the deviation values corresponding to each feature point as the final deviation value.

[0186] The roller bed moves forward, the encoder in the roller bed continuously records the pulse sequence number, the camera is turned on, and continues to acquire images at the preset frame rate;

[0187] When stopping for the second time,

[0188] The camera is turned off, record the current pulse sequence number of the encoder as A’2 = 10000, obtain the translation distance L2 of the current position of the object to be measured compared to the initial position, and the included angle θ2 between the forward direction of the object to be measured and the forward direction of the roller bed;

[0189] Calculate ;

[0190] Calculate the rotation and translation relationship R'' between the coordinate system of the object to be measured and the camera coordinate system during the camera image acquisition period from pulse sequence number A0 to A’2 i T'' i :

[0191] Rotation matrix R'' i = R0, translation matrix ;

[0192] Among them, i represents the i-th image, and S'' = D2×(A i - A0)

[0193] represents the pulse sequence number recorded by the encoder when the i-th image is collected;

[0194] Using R'' i T'' i to solve the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates;

[0195] Calculate the deviation value between the measured coordinates and the coordinates of the feature points in the digital model;

[0196] For example, when stopping this time, a total of 10 images are collected, there are two feature points, which are distributed in the 3rd, 5th, and 8th images. Then, use R''3T''3, R''5T''5, and R''8T''8 to solve the measured coordinates and deviation values;

[0197] Take the average value of the deviation values corresponding to each feature point as the final deviation value.

[0198] The roller bed moves forward, the encoder in the roller bed continuously records the pulse sequence number, the camera is turned on, and continues to collect images at the preset frame rate; until the object to be measured on the roller bed is translated out of the detection area, execute step S3;

[0199] S3. From the two deviation values calculated in step S2, select D1 and θ1 corresponding to the minimum deviation value, and denote them as D and θ respectively;

[0200] Use them to calculate the rotation and translation relationship R i T i :

[0201] Rotation matrix R i = R0, translation matrix ;

[0202] Among them, i represents the i-th image, and S = D×(A i - A0)

[0203] As a preferred implementation manner, the following steps are further included:

[0204] I. In the interval [D - δ, D + δ], take values multiple times according to the preset step size (0.1δ ~ 0.5δ). After each value is taken, the following processing is performed:

[0205] Record the currently obtained value as D';

[0206] Use D' to calculate the translation matrix ;

[0207] Wherein, S' = D'×(A i - A0)

[0208] Using R i T i ' to calculate the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates;

[0209] Calculate the deviation value between the measured coordinates and the coordinates of the feature points in the digital model;

[0210] II. Select the value of D' corresponding to the minimum deviation value, and denote the R i T i ' obtained by solving this value as the final R i T i .

[0211] Wherein, δ takes values in the range of 0.5 mm to 5 mm.

[0212] More specifically, the method for calculating the measured coordinates is as follows:

[0213] Use the pixel coordinates of the camera optical center and the feature points in the image to construct a spatial line L;

[0214] Using R i T i ' to convert the coordinate system of the digital model of the object to be measured to the camera coordinate system, and the spatial line L intersects with the converted digital model of the object to be measured, and denote the intersection coordinates as the three-dimensional coordinates of the feature points in the camera coordinate system;

[0215] Then use R i T i ' to convert the three-dimensional coordinates to the coordinate system of the object to be measured to obtain the measured coordinates.

[0216] Wherein, in order to increase the binding force, select the most reasonable value of D', and preferably implement it as:

[0217] There are multiple feature points distributed in different images. For each feature point, first find the R i T i ' corresponding to the image where it is located, and then use the found R i T i ' to calculate the measured coordinates and deviation values;

[0218] Take the average of the deviation values corresponding to each feature point as the final deviation value.

[0219] In this embodiment, in view of the situation that the object to be measured slips on the roller bed, the value of D is calibrated multiple times, and the value for the best three-dimensional positioning (with the smallest positioning error) is selected from them. At the same time, in order to eliminate the error introduced during the calculation, D is iterated step by step, and the value that satisfies the best three-dimensional positioning (with the smallest positioning error) is selected to improve the accuracy of the external parameter calculation.

[0220] The foregoing description of the specific exemplary embodiments of the present invention has been presented for purposes of illustration and description. The foregoing description is not intended to be exhaustive nor to limit the invention to the precise forms disclosed, and obviously many modifications and variations are possible in light of the above teaching. The exemplary embodiments were chosen and described in order to explain the particular principles of the invention and its practical application to enable others skilled in the art to realize and utilize the various exemplary embodiments of the invention and their various alternative forms and modifications. The scope of the invention is intended to be defined by the appended claims and their equivalents.

Claims

1. A method for real-time acquisition of external camera parameters for a moving object to be measured. The object to be measured is placed on a roller bed, and the camera is fixed around the roller bed. An inductor is fixed around the camera, and the inductor is used to sense whether the object to be measured is in place; The roller bed is provided with rollers, and the rollers are controlled by a motor. The motor is provided with an encoder. When the motor drives the rollers to rotate at a constant speed, the encoder continuously records the pulse number, and the roller bed moves forward; when the motor controls the rollers not to rotate, the encoder stops counting and the roller bed stops; During detection, the roller bed moves forward, and a batch of objects to be measured are sequentially placed on the roller bed and perform continuous translational motion relative to the camera; when a single object to be measured triggers the sensor, the camera starts to collect images of the object to be measured during the motion multiple times at a preset frame rate, where Only a partial area of the object to be measured is included in a single image. As the object to be measured is translated, multiple collected images can cover the entire area of the object to be measured from head to tail; It is characterized in that: before the formal detection starts, one of the objects to be measured is used to perform the following steps to obtain the relative position relationship between the coordinate system of the object to be measured and the camera coordinate system at different times. The X-axis of the coordinate system of the object to be measured is parallel to the translation direction of the roller bed, and the Y-axis is perpendicular to the translation direction: 1) When the object to be measured triggers the sensor, the roller bed stops, and the pulse number recorded by the encoder at this time is extracted and denoted as; Denote this position as the initial position and perform camera calibration to obtain the rotation and translation relationship R0T0 between the coordinate system of the object to be measured and the camera coordinate system; where, R0 represents the rotation matrix, T0 represents the translation matrix, which contains translation components T x0 , T y0 , T z0 ; 2) The roller bed moves forward, and the encoder in the roller bed continuously records the pulse number; Each time the camera captures an image, the pulse number recorded by the encoder is extracted, and the rotation and translation relationship R between the coordinate system of the object to be measured and the camera coordinate system at this moment is calculated using the following formula i T i : Rotation matrix R i = R0, translation matrix ; where i represents the i-th image, S = D×(A i - A0); A i is the pulse number recorded by the encoder when collecting the i-th image, D is the translation distance of the object to be measured between two adjacent pulse numbers calibrated in advance, and θ is the angle between the advance direction of the object to be measured and the advance direction of the roller bed calibrated in advance; Store the calculated R i T i as the real-time external camera parameters; 3) Use the following steps to obtain the final external camera parameters: I. In the interval [D - δ, D + δ], take values multiple times at a preset step length. After each value is taken, the following processing is performed: Record the currently obtained value as D'; Calculate the translation matrix using D' ; where S' = D'×(A i - A0); Utilize R i T i Solve the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates; Calculate the deviation value between the measured coordinate and the coordinate of the feature point in the digital model; II. Select the value of D' corresponding to the minimum deviation value, and calculate the R obtained by solving this value i T i ' and record it as the final R i T i .

2. The method for real-time acquisition of the external camera parameters for the to-be-measured moving object according to claim 1, wherein: The pre-calibration process of D and θ is as follows: When the object to be measured triggers the inductor, the laser tracker obtains the three-dimensional coordinates of the target ball pre-fixed on the object to be measured; Let the roller bed move forward and stop after a preset time. At this time, the laser tracker obtains the three-dimensional coordinates of the marked point on the object to be measured again and records the pulse number recorded by the encoder at this time as A'; Record the included angle between the connection line between the two obtained three-dimensional coordinates and the X-axis as θ; Calculate the spatial distance between the two three-dimensional coordinates and record it as L; Calculation: , where L is the spatial distance between the three-dimensional coordinates obtained twice.

3. A method for real-time acquisition of external camera parameters for a moving object to be measured, characterized in that: The object to be measured is placed on a roller bed, and the camera is fixed around the roller bed. An inductor is fixed around the camera, and the inductor is used to sense whether the object to be measured is in place; The roller bed is provided with rollers, and the rollers are controlled by a motor. The motor is provided with an encoder. When the motor drives the rollers to rotate at a constant speed, the encoder continuously records the pulse number, and the roller bed moves forward; when the motor controls the rollers not to rotate, the encoder stops counting and the roller bed stops; During detection, the roller bed moves forward, and a batch of objects to be measured are sequentially placed on the roller bed and perform continuous translational motion relative to the camera; when a single object to be measured triggers the inductor, the camera starts to collect images of the object to be measured during the motion multiple times at a preset frame rate. Among them, only a partial area of the object to be measured is included in a single image. As the object to be measured is translated, multiple collected images can cover the entire area of the object to be measured from head to tail; It is characterized in that: before the formal detection starts, one of the objects to be measured is used to perform the following steps to obtain the relative position relationship between the coordinate system of the object to be measured and the camera coordinate system at different times. The X-axis of the coordinate system of the object to be measured is parallel to the translation direction of the roller bed, and the Y-axis is perpendicular to the translation direction: S1. When the object to be measured triggers the sensor, the roller bed stops, and the pulse sequence number recorded by the encoder at this time is extracted and denoted as A0; this position is recorded as the initial position and camera calibration is performed to obtain the rotation and translation relationship R0T0 between the coordinate system of the object to be measured and the camera coordinate system; where R0 represents the rotation matrix and T0 represents the translation matrix, which includes translation components T x0 , T y0 , T z0 ; Use the laser tracker to obtain the three-dimensional coordinates Q of the target ball pre-fixed on the object to be measured; S2. The roller bed moves forward, and the encoder in the roller bed continuously records the pulse sequence number. The camera acquires images at a preset frame rate. During the forward movement of the roller bed, the roller bed is randomly stopped multiple times, and step B is performed each time it stops. When the object to be measured is translated out of the detection area, step S3 is executed; The said step B is as follows: The camera is turned off, record the pulse number currently recorded by the encoder as, and obtain the translation distance L of the current position of the object to be measured relative to the initial position. j ; Use the laser tracker to obtain the three-dimensional coordinates Q of the target ball on the object to be measured again. j , and the three-dimensional coordinates Q j The included angle between the line connecting the three-dimensional coordinates Q and the X-axis is denoted as θ. j , θ j represents the included angle between the advancing direction of the object to be measured and the advancing direction of the roller bed; j represents the number of times step B is executed, j = 1, 2, 3... M, and M represents the total number of times. Calculation ; Calculate the rotation and translation relationship R'' j T'' i between the coordinate system of the object to be measured and the camera coordinate system during the camera image acquisition period from pulse number A0 to A i : Rotation matrix R'' i = R0, translation matrix ; where i represents the i-th image, and S'' = D j × (A i - A0); A i denotes the pulse sequence number recorded by the encoder when the i-th image is acquired; Using R'' i T'' i Solve for the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates; Calculate the deviation value between the measured coordinates and the coordinates of the feature point in the digital model; The roller bed moves forward, and the encoder in the roller bed continuously records the pulse sequence number. The camera is turned on and continues to acquire images at a preset frame rate; S3. From the multiple deviation values calculated in step S2, select D corresponding to the minimum deviation value j and θ j , and denote them as D and θ respectively; Calculate the rotation and translation relationship R i T i : Rotation matrix R i = R0, translation matrix ; where i represents the i-th image, and S = D×(A i - A0); Store the calculated R i T i as the real-time extrinsic camera parameters.

4. The method for real-time acquisition of the external camera parameters for the to-be-measured moving object according to claim 3, wherein: The distance L by which the current position of the analyte to be measured is translated compared to the initial position j The methods include the following two: Method 1: Denote the spatial distance between the three-dimensional coordinate Q j and the three-dimensional coordinate Q as L j ; Method 2: In step S1, the camera acquires the surface image of the object to be measured and extracts the pixel coordinates (u, v) of the surface feature points of the object to be measured therein. The three-dimensional coordinates (X1, Y1, Z1) of this feature point are searched on the digital model of the object to be measured; In step B, in the image newly acquired by the camera, the pixel coordinates (u', v') of another feature point are extracted, and the three-dimensional coordinates (X1', Y1', Z1') of this feature point are searched on the digital model of the object to be measured; , where s represents the pixel equivalent.

5. The method for real-time acquisition of the external camera parameters for the to-be-measured moving object according to claim 3, wherein: In step S2, during the forward movement of the roller bed, the roller bed is stopped at equal intervals multiple times.

6. The method for real-time obtaining of the external camera parameters for the to-be-measured moving object according to claim 3, characterized in that: It further includes the following steps: I. In the interval [D - δ, D + δ], take values multiple times according to a preset step size. After each value is taken, the following processing is performed: Record the currently obtained value as D'; Calculate the translation matrix using D' ; where S' = D'×(A i - A0); Using R i T i 'Solve for the coordinates of the feature points in the i-th image in the coordinate system of the object to be measured, denoted as the measured coordinates; Calculate the deviation value between the measured coordinates and the coordinates of the feature point in the digital model; II. Select the value of D' corresponding to the minimum deviation value, and calculate the R obtained by solving this value i T i ' and record it as the final R i T i .

7. The method for real-time obtaining of the external camera parameters for the to-be-measured moving object according to claim 6, characterized in that: The method for solving the measured coordinates is: Construct a space straight line L using the pixel coordinates of the camera optical center and the feature point in the image; Using R i T i 'Convert the digital-analog coordinate system of the object to be measured to the camera coordinate system. The spatial straight line L intersects with the digital-analog of the object to be measured after conversion, and record the intersection coordinates as the three-dimensional coordinates of the feature point in the camera coordinate system; Reuse R i T i 'Convert the three-dimensional coordinates to the coordinate system of the object to be measured to obtain the measured coordinates.

8. The method for real-time acquisition of the external camera parameters for the to-be-measured moving object according to claim 6, wherein: In Step I, there are multiple feature points distributed in different images. For each feature point, first find the corresponding R i T i ' of the image where it is located, and then use the found R i T i ' to calculate the measured coordinates and deviation values; Take the average of the deviation values corresponding to each feature point as the final deviation value.

9. The method for real-time acquisition of the external camera parameters for the to-be-measured moving object according to claim 6, wherein: δ is taken as 0.3mm - 1mm, and the preset step size is taken as 0.1δ - 0.5δ.

Images