An automatic calibration method and system based on joint point searching of a total station and a camera

Abstract

The application provides an automatic calibration method and system based on joint point searching of a total station and a camera, the method establishes a two-dimensional coordinate system and a three-dimensional coordinate system respectively with an industrial camera and a total station as origins, extracts two-dimensional positioning information of a to-be-monitored mark point from a two-dimensional image shot by the industrial camera, then calculates three-dimensional positioning information of point position data obtained by dotting with respect to three-dimensional coordinates of the industrial camera by using a Euclidean distance formula, then matches the two-dimensional positioning information and the three-dimensional positioning information, obtains two-dimensional-three-dimensional positioning information pairing of the same to-be-monitored mark point, solves pose parameters of the industrial camera according to the two-dimensional-three-dimensional positioning information pairing, further reversely projects the three-dimensional positioning information to a two-dimensional image, calculates a reverse projection error, if the reverse projection error is less than an error threshold, converts the two-dimensional positioning information into three-dimensional physical coordinates in the three-dimensional coordinate system according to the pose parameters, and completes two-dimensional-three-dimensional joint calibration.

Description

Technical Field

[0001] This invention belongs to the field of tunnel monitoring technology, specifically relating to an automatic calibration method and system based on the combined point finding of a total station and a camera. Background Technology

[0002] In tunnel construction projects, regular health monitoring of the tunnel structure is beneficial for maintaining its safety. Currently, domestic and international tunnel structural health monitoring mainly adopts non-contact monitoring methods, including laser scanning monitoring, total station monitoring, and machine vision monitoring.

[0003] Traditional total station monitoring methods require a high degree of instrument horizontality, which is difficult to guarantee during actual operation, leading to measurement errors due to attitude errors. In practical engineering applications, laser scanning monitoring methods often suffer from millimeter-level fixed distance measurement errors compared to traditional total stations, especially in environments with short measurement ranges (such as tunnels). Furthermore, machine vision monitoring methods, such as those using industrial cameras, in tunnel environments often suffer from missing common fields of view between cameras due to tunnel corners, resulting in partial loss of deformation information of the monitored object and thus reducing monitoring accuracy. Summary of the Invention

[0004] This invention provides an automatic calibration method and system based on the combined point finding of a total station and a camera, which is used to solve the problems existing in the prior art.

[0005] The first method, as described in this invention, provides an automatic calibration method based on joint point finding using a total station and a camera to achieve the above objectives, comprising:

[0006] S10: Establish a two-dimensional coordinate system with the industrial camera as the origin, and simultaneously establish a three-dimensional coordinate system with the total station as the origin;

[0007] S20: An industrial camera is used to capture images of the marker points to be monitored, resulting in two-dimensional images. The two-dimensional positioning information of the marker points to be monitored is then extracted from the two-dimensional images.

[0008] S30: A total station is used to mark the points to be monitored, and the point data of the points to be monitored is obtained. The three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera is calculated using the Euclidean distance formula.

[0009] S40: Match the two-dimensional and three-dimensional positioning information to obtain a pair of two-dimensional and three-dimensional positioning information for the same marker point to be monitored;

[0010] S50: Based on the pairing of 2D-3D positioning information, the pose of the industrial camera is globally solved to obtain the pose parameters;

[0011] S60: Backproject the three-dimensional positioning information onto the two-dimensional image and calculate the backprojection error;

[0012] S70: Determine whether the back projection error is less than the error threshold. If so, based on the pose parameters, convert the two-dimensional positioning information into three-dimensional physical coordinates in the three-dimensional coordinate system, and perform two-dimensional-three-dimensional joint calibration on the monitoring marker point that has completed the three-dimensional conversion.

[0013] Through the above technical solution, the method of the present invention unifies the monitoring markers detected by industrial cameras and total stations in non-public field of view on the coordinate system, and performs two-dimensional-three-dimensional joint calibration of the monitoring markers. It combines the advantages of total stations and industrial cameras for tunnel monitoring, and overcomes the monitoring defects of using only total stations or industrial cameras for monitoring.

[0014] Preferably, after step S70, the method of the present invention further includes:

[0015] If the back projection error is greater than or equal to the error threshold, the RANSAC algorithm is used to filter out the mismatched two-dimensional and three-dimensional positioning information, and the process returns to step S50.

[0016] Preferably, the two-dimensional positioning information includes at least the positioning coordinates of the marker to be monitored and the pixel radius of the marker to be monitored. Step S20 includes:

[0017] S21: An industrial camera is used to capture images of the marker points to be monitored, and the captured two-dimensional images are stored in the algorithm processing base station;

[0018] S22: The built-in target detection algorithm of the base station is used to select the light spots in the two-dimensional image, and obtain the pixel coordinates of each light spot and the ROI (Region of Interest) of the light spot distribution in the two-dimensional image.

[0019] S23: Based on the ROI of each spot distribution and the pixel coordinates of each spot in the two-dimensional image, calculate the centroid pixel coordinates and numerical radius of each spot;

[0020] S24: Use the centroid pixel coordinates as the positioning coordinates of the marker to be monitored, and use the numerical radius as the pixel radius of the marker to be monitored.

[0021] Preferably, after step S22, the method of the present invention further includes:

[0022] S201: Calculate the spot threshold for each spot based on the image pixel area of ​​each spot in the two-dimensional image;

[0023] S202: Divide the spot with ROI greater than or equal to the corresponding spot threshold into the actual spot region.

[0024] Preferably, the three-dimensional positioning information includes the physical distance from the marker point to be monitored to the industrial camera, and step S40 includes:

[0025] S41: Calculate the pixel magnification factor of a two-dimensional image;

[0026] S42: Based on pixel magnification, establish a pairing relationship model between the pixel radius and physical size distance of the marker to be monitored;

[0027] S43: Randomly insert multiple two-dimensional positioning information and multiple three-dimensional positioning information into the pairing relationship model, and perform information matching on multiple marker points to be monitored in the two-dimensional coordinate system and the three-dimensional coordinate system to obtain multiple two-dimensional-three-dimensional positioning information pairs.

[0028] Preferably, before step S43, the method of the present invention further includes:

[0029] S400: Based on the pixel radius of multiple light spots, multiple groups of monitoring markers located on the same cross section are screened out;

[0030] S401: When placing multiple two-dimensional positioning information and multiple three-dimensional positioning information into the pairing relationship model, the number of monitoring markers located on the same cross section is limited to less than or equal to 2.

[0031] Preferably, after step S400, the method of the present invention further includes:

[0032] Based on multiple sets of monitoring markers located on the same cross section, the curvature of the half-section of the tunnel is approximated as 0.

[0033] Preferably, the point data is the three-dimensional physical coordinates of the marker point to be monitored in a three-dimensional coordinate system, and the three-dimensional coordinates of the industrial camera are the three-dimensional physical coordinates of the industrial camera in a three-dimensional coordinate system. The specific formula for calculating the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera using the Euclidean distance formula in step S30 is as follows:

[0034]

[0035] Among them, X ii It is the X-axis coordinate of the i-th monitoring marker in the three-dimensional coordinate system, and the Y-axis coordinate is... ii Z is the Y-axis coordinate of the i-th monitoring marker in the three-dimensional coordinate system. ii X is the Z-axis coordinate of the i-th monitoring marker in the three-dimensional coordinate system; jj It is the X-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, Y... jjIt is the Y-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, Z... jj Dis is the Z-axis coordinate of the jjth industrial camera in a three-dimensional coordinate system. ii,jj The physical distance between the ii-th monitoring marker and the jj-th industrial camera is used as the 3D positioning information.

[0036] Preferably, after photographing the marker point to be monitored using an industrial camera, the method of the present invention further includes:

[0037] Adjust the position and orientation of the industrial camera until the marker to be monitored is centered in the field of view of the industrial camera.

[0038] Secondly, to achieve the above objectives, the present invention also provides an automatic calibration system based on joint point finding using a total station and a camera, comprising:

[0039] The coordinate establishment module is used to establish a two-dimensional coordinate system with the industrial camera as the origin, and at the same time establish a three-dimensional coordinate system with the total station as the origin;

[0040] The two-dimensional positioning information acquisition module is used to take pictures of the marker point to be monitored using an industrial camera, obtain two-dimensional images, and extract the two-dimensional positioning information of the marker point to be monitored from the two-dimensional images.

[0041] The three-dimensional positioning information acquisition module is used to use a total station to mark the marker points to be monitored, obtain the point data of the marker points to be monitored, and use the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera.

[0042] The information matching module is used to match two-dimensional and three-dimensional positioning information to obtain two-dimensional and three-dimensional positioning information matching for the same monitoring point.

[0043] The pose acquisition module is used to perform global solution of the pose of the industrial camera based on the pairing of two-dimensional and three-dimensional positioning information to obtain the pose parameters.

[0044] The error calculation module is used to backproject the three-dimensional positioning information onto the two-dimensional image and calculate the backprojection error.

[0045] The joint calibration module is used to determine whether the back projection error is less than the error threshold. If so, based on the pose parameters, the two-dimensional positioning information is converted into three-dimensional physical coordinates in the three-dimensional coordinate system, and the two-dimensional-three-dimensional joint calibration is performed on the marker point to be monitored after the three-dimensional conversion is completed.

[0046] The present invention has at least the following advantages:

[0047] 1) Compared with the prior art, the method of the present invention unifies the monitoring markers detected by industrial cameras and total stations in non-public field of view on the coordinate system, and performs two-dimensional-three-dimensional joint calibration of the monitoring markers. It combines the advantages of total stations and industrial cameras for tunnel monitoring, and overcomes the monitoring defects of using only total stations or industrial cameras for monitoring.

[0048] 2) Compared with the prior art, the method of the present invention can automatically calculate the light spot threshold and segment the actual area of ​​the light spot according to the light spot threshold. It has a strong threshold self-adaptation capability and can better adapt to the harsh environment in the tunnel. It can maximize the accuracy of identifying blurred and deformed light spots in harsh tunnel environments such as insufficient light, temperature and distance changes.

[0049] 3) Compared with the prior art, the method of the present invention makes full use of the feature that there are enough monitoring markers in the tunnel scene to perform two-dimensional-three-dimensional matching calculations for multiple points before solving the pose of the industrial camera. This reduces the situation of multiple solutions for the pose of the industrial camera due to the lack of matching points, and improves the robustness and accuracy of the industrial camera calibration.

[0050] 4) Compared with the prior art, the method of the present invention combines the specific longitudinal field of view of the industrial camera with the curvature of the tunnel bend (half-section of the pipe). When analyzing the half-section of the pipe, the pipe is straight instead of curved, which is conducive to deriving the distribution law of the marker points of different sections. It is also conducive to combining the fixed setting law of the marker points of each section to distinguish the marker points well, and theoretically reduces the false detection rate of the light spot. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a flowchart of the automatic calibration method based on the joint point finding of a total station and a camera in Embodiment 1 of the present invention.

[0053] Figure 2 This is a monitoring distribution map of the industrial camera and total station in Embodiment 1 of the present invention.

[0054] Figure 3 This is a diagram illustrating the arrangement of the tunnel ring structure and the monitoring markers in Embodiment 1 of the present invention.

[0055] Figure 4 This is a diagram showing the proportional relationship between the images of marker points at different distances by the industrial camera in Embodiment 1 of the present invention.

[0056] Figure 5 This is an equivalent diagram of the monitoring distribution of industrial cameras and total stations when the curvature of a half-section of the tunnel is approximately 0 in Embodiment 1 of the present invention.

[0057] Figure 6 This is a schematic diagram of the automatic calibration system based on the combined point finding of a total station and a camera in Embodiment 2 of the present invention. Detailed Implementation

[0058] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] In the description of this invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.

[0060] This embodiment discloses an automatic calibration method and system based on the joint point finding of a total station and a camera, which is used to solve the problems existing in the prior art.

[0061] Example 1

[0062] like Figure 1 As shown, the automatic calibration method based on joint point finding using a total station and camera in this embodiment includes:

[0063] S10: Establish a two-dimensional coordinate system with the industrial camera as the origin, and simultaneously establish a three-dimensional coordinate system with the total station as the origin.

[0064] In this embodiment, before step S10, as follows: Figure 2 As shown, industrial cameras, total stations, algorithm processing base stations, and monitoring markers need to be pre-positioned according to the tunnel conditions. Then, a camera coordinate system (two-dimensional coordinate system) is established with the industrial camera as the origin, and a total station coordinate system (three-dimensional coordinate system) is established with the total station as the origin.

[0065] It should be explained that the industrial camera used to monitor the target points is equipped with a total station prism commonly used in this field. Its imaging characteristic is a luminous circular spot. After grayscale processing and image contrast enhancement, the spot imaging becomes more pronounced, enabling high-precision real-time automatic capture over a large depth of field. The choice between a right-handed and left-handed coordinate system for the two-dimensional coordinate system depends on the rules set for the three-dimensional coordinate system. One axis of the three-dimensional coordinate system is parallel to the straight line representing the pipe section being monitored by the total station, while the planes defined by the other two axes are perpendicular to the straight line representing the pipe section being monitored. The target points are set according to general principles; that is, target points on the same cross-section in the tunnel are set according to a uniform pattern, generally arranged from low to high, from the track bed to the arch waist to the arch crown. Target points with nearly equal tunnel depth information are considered to be on the same cross-section, and the distribution of target points on each cross-section is determined according to the order from low to high. The industrial cameras and algorithm processing base stations are set up according to general principles: two industrial cameras are placed at both ends of the tunnel segment to be monitored, ensuring that as many monitoring markers as possible are distributed within the field of view of the industrial cameras, i.e., as centrally as possible. Two algorithm processing base stations are connected to the two industrial cameras respectively, positioned next to the connected cameras. The total station is set up according to general principles: it is positioned at the center of the tunnel segment to be monitored, and can capture data from both industrial cameras using laser marking.

[0066] S20: An industrial camera is used to capture images of the markers to be monitored, resulting in two-dimensional images. The two-dimensional positioning information of the markers to be monitored is then extracted from the two-dimensional images.

[0067] After using an industrial camera to photograph the marker point to be monitored, the method of the present invention further includes:

[0068] Adjust the position and orientation of the industrial camera until the marker to be monitored is centered in the field of view of the industrial camera.

[0069] Using the above technical solution, an industrial camera can be used to capture images of the distribution of multiple monitoring points. If multiple monitoring points are found to be off-center from the industrial camera's field of view, the position of the industrial camera can be corrected by an intelligent gimbal device or manually until the monitoring points are located in the center of the industrial camera's field of view, thereby improving the accuracy of monitoring.

[0070] Specifically, step S20 above includes:

[0071] S21: An industrial camera is used to capture images of the marker points to be monitored, and the captured two-dimensional images are stored in the algorithm processing base station.

[0072] S22: The built-in target detection algorithm of the base station is used to select the light spots in the two-dimensional image, and obtain the pixel coordinates of each light spot and the ROI (Region of Interest) of the light spot distribution in the two-dimensional image.

[0073] S23: Based on the ROI of each spot distribution and the pixel coordinates of each spot in the two-dimensional image, calculate the centroid pixel coordinates and numerical radius of each spot.

[0074] In this embodiment, the specific calculation formula for step S23 is as follows:

[0075]

[0076] Among them, S k Let u be the image pixel area of ​​the k-th spot. p,k Let v be the x-coordinate of the centroid of the k-th spot in a two-dimensional coordinate system. p,k Let H be the ordinate of the centroid of the k-th spot in a two-dimensional coordinate system (pixel coordinates). k (i,j) represents the two-dimensional pixel coordinates of the k-th spot in the two-dimensional coordinate system, where i is the two-dimensional horizontal coordinate value, j is the two-dimensional vertical coordinate value, and j is the numerical radius of the k-th spot.

[0077] S24: Use the centroid pixel coordinates as the positioning coordinates of the marker to be monitored, and use the numerical radius as the pixel radius of the marker to be monitored.

[0078] The two-dimensional positioning information of each marker point to be monitored in the two-dimensional image can be calculated through the above steps S21-S24.

[0079] It should be noted that, after step S22, the method of the present invention further includes:

[0080] S201: Calculate the spot threshold for each spot based on the image pixel area of ​​each spot in the two-dimensional image;

[0081] S202: Divide the spot with ROI greater than or equal to the corresponding spot threshold into the actual spot region.

[0082] Specifically, the calculation formula for step S201 is as follows:

[0083]

[0084] thr k =A k +ind k (6)

[0085] Among them, A k Let thr be the average pixel value of the k-th light spot in the image.k Let be the threshold value corresponding to the k-th spot. k The image statistic for the k-th spot can be either the mode or the standard deviation, depending on the specific tunnel scene and the corresponding target detection performance.

[0086] Through the above steps S201-S202, the method of the present invention can automatically calculate the light spot threshold and segment the actual area of ​​the light spot according to the light spot threshold. It has a strong threshold self-adaptation capability and can better adapt to the harsh environment in the tunnel. It can maximize the accuracy of identifying blurred and deformed light spots in harsh tunnel environments such as insufficient light, temperature and distance changes.

[0087] S30: A total station is used to mark the points to be monitored, obtaining the point location data of the points to be monitored, and the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera is calculated using the Euclidean distance formula.

[0088] In this embodiment, a total station is used to... Figure 2 The marker points monitored by the two industrial cameras are marked. The point location data is the three-dimensional physical coordinate of the marker point in the three-dimensional coordinate system, and the three-dimensional coordinate of the industrial camera is the three-dimensional physical coordinate of the industrial camera in the three-dimensional coordinate system. The specific formula for calculating the three-dimensional positioning information of the point location data relative to the three-dimensional coordinate of the industrial camera using the Euclidean distance formula in step S30 is as follows:

[0089]

[0090] Among them, X ii It is the X-axis coordinate of the i-th monitoring marker in the three-dimensional coordinate system, and the Y-axis coordinate is... ii Z is the Y-axis coordinate of the i-th monitoring marker in the three-dimensional coordinate system. ii X is the Z-axis coordinate of the i-th monitoring marker in the three-dimensional coordinate system; jj It is the X-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, Y... jj It is the Y-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, Z... jj Dis is the Z-axis coordinate of the jjth industrial camera in a three-dimensional coordinate system. ii,jj The physical distance between the ii-th monitoring marker and the jj-th industrial camera is used as the 3D positioning information.

[0091] S40: Match the two-dimensional and three-dimensional positioning information to obtain a pair of two-dimensional and three-dimensional positioning information for the same monitoring point.

[0092] Specifically, step S40 includes:

[0093] S41: Calculate the pixel magnification factor of a two-dimensional image;

[0094] S42: Based on pixel magnification, establish a pairing relationship model between the pixel radius and physical size distance of the marker to be monitored;

[0095] S43: Randomly insert multiple two-dimensional positioning information and multiple three-dimensional positioning information into the pairing relationship model, and perform information matching on multiple marker points to be monitored in the two-dimensional coordinate system and the three-dimensional coordinate system to obtain multiple two-dimensional-three-dimensional positioning information pairs.

[0096] It should be explained that the above steps S41-S43 are used to realize the one-to-one correspondence between two-dimensional positioning information and three-dimensional positioning information by utilizing the inherent laws of the physical size distance between the monitored marker and the industrial camera and the pixel radius of the monitored marker in the image, thereby automatically constructing a two-dimensional-three-dimensional positioning information matching pair for the marker.

[0097] Further explanation is needed, such as Figure 3 As shown, the circular structure of the tunnel and the arrangement of the monitoring markers exhibit strong regularity in the industrial camera's image. The size of each marker in the image is directly related to the straight-line distance between the industrial camera and the marker, with the corresponding proportional relationship as follows: Figure 4 As shown, steps S41-S43 can be obtained using the above rules. Among them, Figure 4 The formula for the proportional relationship shown is as follows:

[0098]

[0099] Where H1 and H2 are the actual physical dimensions, d1 and d2 are the physical dimensions of the image, f is the camera focal length, and L is the focal length. k,1 Let be the vertical distance from the optical center of the k-th industrial camera to the tunnel cross-section where the marker to be monitored is located. Based on the aforementioned proportional relationship, the following equation can be derived:

[0100]

[0101] From the above proportional relationship (8), it can be seen that the actual physical size ΔH and the image physical size Δd differ only from f and L. k,1 The relevant fixed scaling factor, ΔH, is the diameter of the marker to be monitored. Since this diameter is fixed and known, the physical size of the marker to be monitored in the image within the same cross-section is also fixed.

[0102] Based on the proportional relationships (8) and (9), more specifically, the specific formula for step S41 above is:

[0103]

[0104] Where d is the pixel size of the industrial camera, and L k+1,1 K represents the vertical distance from the optical center of the (k+1)th industrial camera to the tunnel cross-section where the monitored marker is located. k+1 This represents the pixel magnification factor (unit: mm / pixel) of the (k+1)th industrial camera 2D image.

[0105] Taking measurement errors into account, the pairing relationship model in step S42 is as follows:

[0106]

[0107]

[0108] Wherein, formula (13) represents the correspondence between the pixel radius in the two-dimensional positioning information of the marker to be monitored and the physical size distance D in the three-dimensional positioning information of the marker to be monitored, and formulas (11) and (12) are the constraints of formula (13).

[0109] It should be explained that the pixel size and focal length of the industrial camera are constant for different marker points to be monitored. Therefore, the main factor affecting the constraint of formula (11) is the vertical distance L from the optical center of the industrial camera to the tunnel section where the marker point to be monitored is located. k+1,1 This allows for the use of tunnel depth information as a criterion to distinguish imaging spots. Generally, the longitudinal field of view of each industrial camera is between 10m and 200m; therefore, the three-dimensional positioning information of the industrial camera in the three-dimensional coordinate system (Dis) is crucial. ii,jj The tunnel cross-sectional radius R' must satisfy the constraint formula (12). Randomly place multiple two-dimensional positioning information and multiple three-dimensional positioning information into the pairing relationship model. If a certain pair of two-dimensional positioning information and three-dimensional positioning information satisfies the above relationship formulas (11)-(13), it means that the pair of two-dimensional positioning information and three-dimensional positioning information are matched with each other and form a two-dimensional-three-dimensional positioning information pair. Such two-dimensional-three-dimensional positioning information pair can be stored using a database linked list structure.

[0110] Preferably, prior to step S43 above, the method of the present invention further includes:

[0111] S400: Based on the pixel radius of multiple light spots, multiple groups of monitoring markers located on the same cross section are screened out;

[0112] S401: When placing multiple two-dimensional positioning information and multiple three-dimensional positioning information into the pairing relationship model, the number of monitoring markers located on the same cross section is limited to less than or equal to 2.

[0113] Compared with existing technologies, the method of this invention can screen the monitoring markers on different tunnel cross sections, limiting the number of monitoring markers located on the same cross section to less than or equal to 2, which can avoid model solution singularities and overcome the influence of cross sections with different depths when using industrial camera two-dimensional image monitoring.

[0114] Preferably, after step S400, the method of the present invention further includes:

[0115] Based on multiple sets of monitoring markers located on the same cross section, the curvature of the half-section of the tunnel is approximated as 0.

[0116] Specifically, such as Figure 5 As shown, the monitoring markers of a certain tunnel section can be approximated as having the same camera tunnel depth information. Under the premise of satisfying formula (12), the half-section of the pipe section monitored by the industrial camera can be approximated as a straight section of pipe section with a curvature of almost 0. The straight direction of the half-section of the pipe section is taken as the direction of the line connecting the geometric center of the first end section and the geometric center of the last end section of the pipe section.

[0117] Through the above technical solution, the method of the present invention combines the specific longitudinal field of view of the industrial camera with the curvature of the tunnel bend (half-section of the pipe). When analyzing the half-section of the pipe, the pipe is straight instead of curved, which is conducive to deriving the distribution law of the marker points of different sections. It is also conducive to combining the fixed setting law of the marker points of each section to distinguish the marker points well, and theoretically reduces the false detection rate of the light spot.

[0118] Following step S40 above, the method of the present invention further includes:

[0119] S50: Based on the pairing of 2D and 3D positioning information, the pose of the industrial camera is solved globally to obtain the pose parameters.

[0120] In this embodiment, assuming the marker point to be monitored has a rotation matrix R” and a translation vector T” for converting its three-dimensional physical coordinates in the three-dimensional coordinate system to two-dimensional pixel coordinates in the two-dimensional coordinate system, then their expressions are:

[0121]

[0122] Based on the principle of central projection, the two-dimensional pixel coordinates and three-dimensional physical coordinates of each marker point to be monitored can be used to formulate a system of equations:

[0123]

[0124] Among them, C x and C y F represents the principal point of a two-dimensional image, specifically the two-dimensional pixel coordinates of the intersection of the optical axis and the image plane. x and F yGiven the equivalent focal lengths in the x and y directions of the two-dimensional coordinate system, the two-dimensional-three-dimensional positioning information matching pairs of each marker point to be monitored can be listed as a set of equations as shown in formula (15). There are 12 parameters involved in the calculation of formulas (14)-(15), including the camera's intrinsic parameters, namely C. x C y F x F y Given the parameters, the parameters r'0'-r'8' in the rotation matrix and the translation vector T' are... x '、T' y '、T' z ' is the value to be solved. To solve the pose of the above industrial camera, combined with the inherent quantitative relationship between the parameters R” and T”, it is necessary to substitute 6 or more sets of two-dimensional and three-dimensional positioning information matching pairs of the markers to be monitored into formulas (14)-(15) to obtain a unique solution.

[0125] S60: Backproject the 3D positioning information onto the 2D image and calculate the backprojection error.

[0126] Specifically, the calculation formula for step S60 above is as follows:

[0127]

[0128] Where N is the total number of light spots in the two-dimensional image; u′ p,k Let v′ be the two-dimensional abscissa of the k-th marker point to be monitored, projected from the three-dimensional positioning information to the two-dimensional image. p,k The two-dimensional ordinate of the k-th marker point to be monitored is the result of back-projecting the three-dimensional positioning information onto the two-dimensional image.

[0129] S70: Determine whether the back projection error is less than the error threshold. If so, based on the pose parameters, convert the two-dimensional positioning information into three-dimensional physical coordinates in the three-dimensional coordinate system, and perform two-dimensional-three-dimensional joint calibration on the monitoring marker point that has completed the three-dimensional conversion.

[0130] Specifically, if the back projection error is less than the error threshold, and the half-section of the tunnel segment monitored by the industrial camera is approximately considered as a straight section with a curvature of almost zero, it indicates that the imaging spot coordinates of the marker point to be monitored in the images of the two industrial cameras can be unified, and it is assumed that the change along the extension direction of the tunnel segment is small and negligible. Based on the rotation matrix and translation vector from the two industrial cameras to the total station obtained by formulas (14) and (15), the conversion formulas for the two-dimensional pixel coordinates and three-dimensional physical coordinates of the marker point to be monitored, ignoring the change along the extension direction of the tunnel segment, can be derived:

[0131]

[0132] In formula (17), the numbers 1 and 2 appearing in the subscript represent industrial camera 1 and industrial camera 2 monitoring the same marker point. The formula for unifying the two-dimensional image coordinates acquired by the two industrial cameras into three-dimensional physical coordinates in the total station coordinate system is as follows:

[0133]

[0134] The joint calibration of the joint monitoring system can be achieved using formula (18).

[0135] It should be noted that when the marker to be monitored undergoes displacement, the expression for calculating the marker in the total station coordinate system is as follows:

[0136]

[0137] Among them, (X′ k,1 ,Y′ k,1 ,1) and (X′ k,2 ,Y′ k,2 ,1) represents the coordinates of the same monitored marker point in the three-dimensional coordinate system after displacement by two industrial cameras, Δx is the X-axis offset of the monitored marker point in the two-dimensional image, and Δy is the Y-axis offset of the monitored marker point in the two-dimensional image.

[0138] Through steps S10-S70 above, compared with the prior art, the method of the present invention unifies the monitoring marker points detected by industrial cameras and total stations in non-common field of view on the coordinate system, and performs two-dimensional-three-dimensional joint calibration of the monitoring marker points. It combines the advantages of both total stations and industrial cameras for tunnel monitoring, and overcomes the monitoring defects of using only total stations or industrial cameras. Furthermore, before solving for the pose of the industrial camera, the method of the present invention fully utilizes the characteristic of a sufficient number of monitoring marker points in the tunnel scene to perform multi-point two-dimensional-three-dimensional matching calculations, reducing the situation of multiple solutions for the industrial camera pose due to a lack of matching points, and improving the robustness and accuracy of the industrial camera calibration.

[0139] Preferably, after step S70, the method of the present invention further includes:

[0140] If the back projection error is greater than or equal to the error threshold, the RANSAC algorithm is used to filter out the mismatched two-dimensional and three-dimensional positioning information, and the process returns to step S50.

[0141] Furthermore, the method of the present invention uses the RANSAC framework to filter out points with low matching accuracy or mismatches one by one, and lists the filtered point set into a system of equations for solving until the reprojection error is less than a set threshold or the number of iterations exceeds the set limit for the number of iterations, and then ends the RANSAC algorithm.

[0142] Compared with existing technologies, the method of the present invention can screen out mismatched or poorly matched markers to be detected, thereby improving the accuracy of the two-dimensional-three-dimensional joint calibration of the method of the present invention.

[0143] Example 2

[0144] like Figure 6 As shown, based on Embodiment 1, this embodiment discloses an automatic calibration system for joint point finding using a total station and a camera. The system includes:

[0145] The coordinate establishment module is used to establish a two-dimensional coordinate system with the industrial camera as the origin, and at the same time establish a three-dimensional coordinate system with the total station as the origin;

[0146] The two-dimensional positioning information acquisition module is used to take pictures of the marker point to be monitored using an industrial camera, obtain two-dimensional images, and extract the two-dimensional positioning information of the marker point to be monitored from the two-dimensional images.

[0147] The three-dimensional positioning information acquisition module is used to use a total station to mark the marker points to be monitored, obtain the point data of the marker points to be monitored, and use the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera.

[0148] The information matching module is used to match two-dimensional and three-dimensional positioning information to obtain two-dimensional and three-dimensional positioning information matching for the same monitoring point.

[0149] The pose acquisition module is used to perform global solution of the pose of the industrial camera based on the pairing of two-dimensional and three-dimensional positioning information to obtain the pose parameters.

[0150] The error calculation module is used to backproject the three-dimensional positioning information onto the two-dimensional image and calculate the backprojection error.

[0151] The joint calibration module is used to determine whether the back projection error is less than the error threshold. If so, based on the pose parameters, the two-dimensional positioning information is converted into three-dimensional physical coordinates in the three-dimensional coordinate system, and the two-dimensional-three-dimensional joint calibration is performed on the marker point to be monitored after the three-dimensional conversion is completed.

[0152] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as ROM, RAM, magnetic disk, or optical disk.

[0153] For those skilled in the art, various other corresponding changes and modifications can be made based on the technical solutions and concepts described above, and all such changes and modifications should fall within the protection scope of the claims of this invention.

Claims

1. An automatic calibration method based on combined point finding using a total station and a camera, characterized in that, include: S10: Establish a two-dimensional coordinate system with the industrial camera as the origin, and simultaneously establish a three-dimensional coordinate system with the total station as the origin; S20: Use the industrial camera to capture the marker point to be monitored to obtain a two-dimensional image, and extract the two-dimensional positioning information of the marker point to be monitored from the two-dimensional image; S30: The total station is used to mark the points to be monitored to obtain the point data of the points to be monitored, and the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera is calculated using the Euclidean distance formula. S40: Match the two-dimensional positioning information and the three-dimensional positioning information to obtain a pair of two-dimensional and three-dimensional positioning information for the same monitoring point; S50: Based on the pairing of the two-dimensional-three-dimensional positioning information, the pose of the industrial camera is globally solved to obtain the pose parameters; S60: Back-project the three-dimensional positioning information onto the two-dimensional image, and calculate the back-projection error; S70: Determine whether the back projection error is less than the error threshold. If so, based on the pose parameters, convert the two-dimensional positioning information into three-dimensional physical coordinates in the three-dimensional coordinate system, and perform two-dimensional-three-dimensional joint calibration on the monitoring marker point that has completed the three-dimensional conversion. Wherein, the point data refers to the three-dimensional physical coordinates of the marker point to be monitored in the three-dimensional coordinate system, and the three-dimensional coordinates of the industrial camera refer to the three-dimensional physical coordinates of the industrial camera in the three-dimensional coordinate system. The specific formula for calculating the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera using the Euclidean distance formula in step S30 is as follows: in, It is the first ii The X-axis coordinates of the monitored marker points in the three-dimensional coordinate system. It is the first ii The Y-axis coordinates of the monitored marker points in the three-dimensional coordinate system. It is the first ii The Z-axis coordinates of the monitored marker points in the three-dimensional coordinate system; It is the first jj The X-axis coordinates of the industrial camera in the three-dimensional coordinate system. It is the first jj The Y-axis coordinate of the industrial camera in the three-dimensional coordinate system. It is the first jj The Z-axis coordinates of the industrial camera in the three-dimensional coordinate system; For the first ii The number of monitoring markers to the first jj The physical dimensions and distances of the industrial cameras are used as the three-dimensional positioning information.

2. The automatic calibration method based on joint point finding using a total station and camera as described in claim 1, characterized in that, After step S70, the method further includes: if the back projection error is greater than or equal to the error threshold, using the RANSAC algorithm to filter out the mismatched two-dimensional positioning information and three-dimensional positioning information, and returning to step S50.

3. The automatic calibration method based on joint point finding using a total station and camera as described in claim 1, characterized in that, The two-dimensional positioning information includes at least the positioning coordinates of the marker point to be monitored and the pixel radius of the marker point to be monitored. Step S20 includes: S21: Use the industrial camera to capture images of the marker points to be monitored, and store the captured two-dimensional images in the algorithm processing base station; S22: The built-in target detection algorithm of the base station is used to select the light spots in the two-dimensional image, and the pixel coordinates of each light spot and the ROI of the light spot distribution are obtained in the two-dimensional image. S23: Calculate the centroid pixel coordinates and numerical radius of each spot based on the ROI of each spot distribution and the pixel coordinates of each spot in the two-dimensional image; S24: Use the centroid pixel coordinates as the positioning coordinates of the marker point to be monitored, and use the numerical radius as the pixel radius of the marker point to be monitored.

4. The automatic calibration method based on joint point finding using a total station and camera as described in claim 3, characterized in that, After step S22, the method further includes: S201: Calculate the spot threshold for each spot based on the image pixel area of ​​each spot in the two-dimensional image; S202: Divide the spot with ROI greater than or equal to the corresponding spot threshold into the actual spot region.

5. The automatic calibration method based on joint point finding using a total station and camera as described in claim 3, characterized in that, The three-dimensional positioning information includes the physical distance from the marker point to be monitored to the industrial camera, and step S40 includes: S41: Calculate the pixel magnification factor of the two-dimensional image; S42: Based on the pixel magnification factor, establish a pairing relationship model between the pixel radius of the marker to be monitored and the physical size distance; S43: Randomly insert multiple two-dimensional positioning information and multiple three-dimensional positioning information into the pairing relationship model, and perform information matching on multiple marker points to be monitored in the two-dimensional coordinate system and the three-dimensional coordinate system to obtain multiple two-dimensional-three-dimensional positioning information pairs.

6. The automatic calibration method based on joint point finding using a total station and camera as described in claim 5, characterized in that, Prior to step S43, the method further includes: S400: Based on the pixel radius of multiple light spots, multiple groups of monitoring markers located on the same cross section are screened out; S401: When placing multiple two-dimensional positioning information and multiple three-dimensional positioning information into the pairing relationship model, the number of monitoring markers located on the same cross section is limited to less than or equal to 2.

7. The automatic calibration method based on joint point finding using a total station and camera as described in claim 6, characterized in that, After step S400, the method further includes: Based on multiple sets of monitoring markers located on the same cross section, the curvature of the half-section of the tunnel is approximated as 0.

8. The automatic calibration method based on joint point finding using a total station and camera according to claim 1, characterized in that, After the industrial camera is used to photograph the marker point to be monitored, the method further includes: Adjust the position and orientation of the industrial camera until the marker to be monitored is located in the center of the industrial camera's field of view.

9. An automatic calibration system based on combined point finding using a total station and a camera, characterized in that, The system employs the automatic calibration method based on joint point finding using a total station and camera as described in any one of claims 1-8, wherein the system comprises: The coordinate establishment module is used to establish a two-dimensional coordinate system with the industrial camera as the origin, and at the same time establish a three-dimensional coordinate system with the total station as the origin; The two-dimensional positioning information acquisition module is used to take pictures of the marker point to be monitored using the industrial camera, obtain a two-dimensional image, and extract the two-dimensional positioning information of the marker point to be monitored from the two-dimensional image. The three-dimensional positioning information acquisition module is used to use the total station to mark the target point to be monitored, obtain the point data of the target point, and use the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera. The information matching module is used to match the two-dimensional positioning information and the three-dimensional positioning information to obtain two-dimensional and three-dimensional positioning information matching for the same monitoring marker point; The pose acquisition module is used to perform global solution of the pose of the industrial camera based on the pairing of the two-dimensional and three-dimensional positioning information to obtain pose parameters. An error calculation module is used to back-project the three-dimensional positioning information onto the two-dimensional image and calculate the back-projection error. The joint calibration module is used to determine whether the back projection error is less than the error threshold. If so, based on the pose parameters, the two-dimensional positioning information is converted into three-dimensional physical coordinates in the three-dimensional coordinate system, and two-dimensional-three-dimensional joint calibration is performed on the monitoring marker point that has completed the three-dimensional conversion.

Images