Automatic calibration method and system based on joint point searching of total station and camera

Abstract

The invention provides an automatic calibration method and system based on joint point searching of a total station and a camera, and the method comprises the steps: respectively building a two-dimensional coordinate system and a three-dimensional coordinate system with an industrial camera and the total station as original points, and extracting the two-dimensional positioning information of a to-be-monitored mark point from a two-dimensional image shot by the industrial camera; then calculating three-dimensional positioning information of point location data obtained by dotting relative to three-dimensional coordinates of the industrial camera by using an Euclidean distance formula, then performing information matching on the two-dimensional positioning information and the three-dimensional positioning information to obtain two-dimensional-three-dimensional positioning information pairing of the same mark point to be monitored, and then performing two-dimensional-three-dimensional positioning information pairing according to the two-dimensional-three-dimensional positioning information pairing. And solving a pose parameter of the industrial camera, back-projecting the three-dimensional positioning information to the two-dimensional image, calculating to obtain a back-projection error, and if the back-projection error is smaller than an error threshold value, converting the two-dimensional positioning information into a three-dimensional physical coordinate in a three-dimensional coordinate system according to the pose parameter, thereby completing two-dimensional-three-dimensional joint calibration.

Description

Technical Field

[0001] The present invention belongs to the technical field of tunnel monitoring, and in particular relates to an automatic calibration method and system based on joint point finding of a total station and a camera. Background Art

[0002] In tunnel construction projects, regular health monitoring of tunnel structures is beneficial to maintaining the safety of tunnel structures. At present, the health monitoring of tunnel structures at home and abroad mainly adopts non-contact monitoring methods, mainly including: laser scanning monitoring method, total station monitoring method and machine vision monitoring method.

[0003] The traditional total station monitoring method has high requirements for the instrument's attitude level. It is difficult to completely ensure that the total station is placed horizontally during actual operation, and its attitude error will lead to measurement errors. In actual engineering applications, laser scanning monitoring methods are used. Because laser scanners are used in environments with short measurement ranges (such as tunnels), there are often millimeter-level fixed distance measurement errors compared to traditional total station measurement results. When machine vision monitoring methods represented by industrial cameras are used in tunnel environments, the corners of the tunnel often cause the loss of the common field of view between cameras, resulting in partial loss of deformation information of the monitored object, thereby reducing monitoring accuracy. Summary of the invention

[0004] The present invention provides an automatic calibration method and system based on joint point finding of a total station and a camera, which are used to solve the problems existing in the prior art.

[0005] First method, in order to achieve the above-mentioned purpose, the present invention provides an automatic calibration method based on joint point finding of a total station and a camera, comprising:

[0006] S10: Establish a two-dimensional coordinate system with the industrial camera as the origin, and establish a three-dimensional coordinate system with the total station as the origin;

[0007] S20: photographing the marker point to be monitored using an industrial camera to obtain a two-dimensional image, and extracting two-dimensional positioning information of the marker point to be monitored from the two-dimensional image;

[0008] S30: using a total station to mark the marker points to be monitored, obtaining the point data of the marker points to be monitored, and using the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera;

[0009] S40: Matching the two-dimensional positioning information with the three-dimensional positioning information to obtain a pair of two-dimensional and three-dimensional positioning information for the same marker point to be monitored;

[0010] S50: performing a global solution for the posture of the industrial camera according to the pairing of the two-dimensional and three-dimensional positioning information to obtain posture parameters;

[0011] S60: Back-projecting the three-dimensional positioning information onto the two-dimensional image, and calculating a back-projection error;

[0012] S70: Determine whether the back-projection error is less than the error threshold. If so, based on the posture parameters, convert the two-dimensional positioning information into three-dimensional physical coordinates in the three-dimensional coordinate system, and perform two-dimensional-three-dimensional joint calibration on the marker points to be monitored that have completed the three-dimensional conversion.

[0013] Through the above technical scheme, the method of the present invention unifies the monitored marker points monitored by the industrial camera and the total station in the non-public field of view in the coordinate system, and performs two-dimensional-three-dimensional joint calibration on the monitored marker points, combining the advantages of the total station and the industrial camera for tunnel monitoring, and overcoming the monitoring defects existing in the monitoring by only using the total station or the industrial camera.

[0014] Preferably, after step S70, the method of the present invention further comprises:

[0015] If the back-projection error is greater than or equal to the error threshold, the RANSAC algorithm is used to filter out the mismatched two-dimensional positioning information and three-dimensional positioning information, and the process returns to step S50.

[0016] Preferably, the two-dimensional positioning information at least includes the positioning coordinates of the marker point to be monitored and the pixel radius of the marker point to be monitored, and step S20 includes:

[0017] S21: Use an industrial camera to photograph the marker points to be monitored, and store the photographed two-dimensional images in an algorithm processing base station;

[0018] S22: using the built-in target detection algorithm of the algorithm processing base station to frame the light spot of the two-dimensional image, and obtain the pixel coordinates of each light spot in the two-dimensional image and the ROI (Region of Interest) of the light spot distribution;

[0019] S23: Calculate the centroid pixel coordinates and numerical radius of each light spot according to the ROI of each light spot distribution and the pixel coordinates of each light spot in the two-dimensional image;

[0020] S24: Using the centroid pixel coordinates as the positioning coordinates of the marker point to be monitored, and using the numerical radius as the pixel radius of the marker point to be monitored.

[0021] Preferably, after step S22, the method of the present invention further comprises:

[0022] S201: Calculating a light spot threshold of each light spot based on an image pixel area of ​​each light spot in the two-dimensional image;

[0023] S202: Segment the light spot whose ROI is greater than or equal to the corresponding light spot threshold into the light spot actual area.

[0024] Preferably, the three-dimensional positioning information includes the physical size distance from the marker point to be monitored to the industrial camera, and step S40 includes:

[0025] S41: Calculate the pixel magnification of the two-dimensional image;

[0026] S42: Based on the pixel magnification factor, a pairing relationship model between the pixel radius and the physical size distance of the marker point to be monitored is established;

[0027] S43: randomly placing multiple two-dimensional positioning information and multiple three-dimensional positioning information into a pairing relationship model, matching the information of multiple markers to be monitored in a two-dimensional coordinate system and a three-dimensional coordinate system, and obtaining multiple two-dimensional-three-dimensional positioning information pairs.

[0028] Preferably, before step S43, the method of the present invention further comprises:

[0029] S400: screening out multiple groups of marker points to be monitored located on the same section according to the pixel radius of the multiple light spots;

[0030] S401: When placing a plurality of two-dimensional positioning information and a plurality of three-dimensional positioning information into the pairing relationship model, it is limited that the number of marker points to be monitored located in the same section is less than or equal to 2.

[0031] Preferably, after step S400, the method of the present invention further comprises:

[0032] Based on multiple groups of marker points to be monitored located in the same section, the curvature of the half-section of the tunnel is approximated to 0.

[0033] Preferably, the point data is the three-dimensional physical coordinates of the marker to be monitored in the three-dimensional coordinate system, and the three-dimensional coordinates of the industrial camera are the three-dimensional physical coordinates of the industrial camera in the three-dimensional coordinate system. The specific formula for calculating the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera using the Euclidean distance formula in step S30 is:

[0034]

[0035] Among them, X ii is the X-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system, ii is the Y-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system, and Z ii is the Z-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system; jj is the X-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, Y jjis the Y-axis coordinate of the jj-th industrial camera in the three-dimensional coordinate system, Z jj is the Z-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system; Dis ii,jj is the physical size distance from the ii-th marker point to be monitored to the jj-th industrial camera, which serves as the three-dimensional positioning information.

[0036] Preferably, after photographing the marker point to be monitored using an industrial camera, the method of the present invention further comprises:

[0037] Adjust the position of the industrial camera until the marker point to be monitored is located in the center of the industrial camera's field of view.

[0038] In a second aspect, the present invention further provides an automatic calibration system based on joint point finding of a total station and a camera to achieve the above-mentioned purpose, comprising:

[0039] The coordinate establishment module is used to establish a two-dimensional coordinate system with the industrial camera as the origin, and to establish a three-dimensional coordinate system with the total station as the origin;

[0040] A two-dimensional positioning information acquisition module is used to use an industrial camera to shoot the marker point to be monitored to obtain a two-dimensional image, and extract the two-dimensional positioning information of the marker point to be monitored from the two-dimensional image;

[0041] The three-dimensional positioning information acquisition module is used to use a total station to mark the marker points to be monitored, obtain the point data of the marker points to be monitored, and use the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera;

[0042] An information pairing module is used to match the two-dimensional positioning information with the three-dimensional positioning information to obtain a two-dimensional-three-dimensional positioning information pairing for the same marker point to be monitored;

[0043] The posture acquisition module is used to globally solve the posture of the industrial camera based on the pairing of 2D-3D positioning information to obtain the posture parameters;

[0044] An error calculation module is used to back-project the three-dimensional positioning information onto a two-dimensional image and calculate a back-projection error;

[0045] The joint calibration module is used to determine whether the back-projection error is less than the error threshold. If so, based on the posture parameters, the two-dimensional positioning information is converted into three-dimensional physical coordinates in the three-dimensional coordinate system, and the two-dimensional-three-dimensional joint calibration is performed on the monitored landmarks that have completed the three-dimensional conversion.

[0046] The present invention has at least the following advantages:

[0047] 1) Compared with the prior art, the method of the present invention unifies the marker points to be monitored monitored by the industrial camera and the total station in the non-public field of view in the coordinate system, and performs a two-dimensional-three-dimensional joint calibration on the marker points to be monitored, which combines the advantages of the total station and the industrial camera for tunnel monitoring, and overcomes the monitoring defects existing in the monitoring by only using the total station or the industrial camera.

[0048] 2) Compared with the prior art, the method of the present invention can automatically calculate the spot threshold and segment the actual spot area according to the spot threshold. It has a strong threshold adaptive ability and can better adapt to the harsh environment in the tunnel. It can improve the accuracy of identifying blurred and deformed spots in harsh tunnel environments such as insufficient light, temperature, and distance changes.

[0049] 3) Compared with the prior art, the method of the present invention fully utilizes the feature of a sufficient number of monitored landmarks in the tunnel scene to perform two-dimensional-three-dimensional matching calculations of multiple points before solving the posture of the industrial camera, thereby reducing the situation where the posture of the industrial camera is solved multiple times due to lack of matching points, and improving the robustness and accuracy of the industrial camera calibration.

[0050] 4) Compared with the prior art, the method of the present invention combines the specific longitudinal field of view of the industrial camera with the curvature of the tunnel bend (half-section tube section). When analyzing the half-section tube section, the tube section is straight instead of curved, so as to facilitate the deduction of the distribution law of the marking points of different sections, and facilitate the distinction of the marking points in combination with the fixed setting law of the marking points of each section, thereby theoretically reducing the false detection rate of the light spot. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0052] Figure 1 This is a flow chart of an automatic calibration method based on joint point finding by a total station and a camera in Embodiment 1 of the present invention.

[0053] Figure 2 This is a monitoring distribution diagram of the industrial camera and the total station in Example 1 of the present invention.

[0054] Figure 3 This is a diagram showing the arrangement rules of the tunnel ring structure and the marker points to be monitored in the first embodiment of the present invention.

[0055] Figure 4 This is a proportional relationship diagram of the imaging of marker points at different distances by the industrial camera in Example 1 of the present invention.

[0056] Figure 5 This is an equivalent diagram of the monitoring distribution of the industrial camera and the total station when the curvature of the half-section of the tunnel in the first embodiment of the present invention is approximately 0.

[0057] Figure 6 Schematic diagram of the structure of an automatic calibration system based on joint point finding by a total station and a camera in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0058] The technical solution of the present invention will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0059] In the description of the present invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limitations on the present invention.

[0060] This embodiment discloses an automatic calibration method and system based on joint point finding of a total station and a camera, which are used to solve the problems existing in the prior art.

[0061] Embodiment 1

[0062] like Figure 1 As shown, the automatic calibration method based on joint point finding of a total station and a camera in this embodiment includes:

[0063] S10: A two-dimensional coordinate system is established with the industrial camera as the origin, and a three-dimensional coordinate system is established with the total station as the origin.

[0064] In this embodiment, before step S10, Figure 2 As shown, it is necessary to arrange industrial cameras, total stations, algorithm processing base stations and markers to be monitored in advance according to the tunnel conditions, and then establish a camera coordinate system (two-dimensional coordinate system) with the industrial camera as the origin, and establish a total station coordinate system (three-dimensional coordinate system) with the total station as the origin.

[0065] It should be explained that the marker points to be monitored by the industrial camera are hung or installed with the total station prism commonly used in the field, and the imaging feature is a luminous circular light spot. After the image is grayed and the image contrast is enhanced, the light spot imaging is more obvious, and high-precision real-time automatic capture of a large depth of field can be achieved. The direction of the two-dimensional coordinate system is selected as the right-hand coordinate system or the left-hand coordinate system depends on the rules set by the three-dimensional coordinate system. The direction of one axis of the three-dimensional coordinate system is parallel to the direction of the straight line representing the pipe section monitored by the total station, and the plane determined by the other two axes is perpendicular to the direction of the straight line representing the pipe section monitored by the total station. The marker points to be monitored are set according to general principles, that is, the marker points of the same section in the tunnel are set according to a unified rule, generally arranged in order from low to high, from the roadbed to the arch waist to the arch top, and the marker points with nearly equal tunnel depth information are judged as the same section points, and the distribution of each section marker point is determined in order from low to high. The industrial camera and algorithm processing base station are set up according to general principles, specifically: two industrial cameras are placed at both ends of the tunnel section to be monitored, and as many markers to be monitored as possible are distributed within the field of view of the industrial camera, that is, as distributed as possible in the center of the field of view. The two algorithm processing base stations are connected to the two industrial cameras respectively and are located next to the connected industrial cameras. The total station is set up according to general principles, specifically: the total station is set in the center of the tunnel section to be monitored, and both industrial cameras can be captured by laser dot marking.

[0066] S20: photographing the marker point to be monitored using an industrial camera to obtain a two-dimensional image, and extracting two-dimensional positioning information of the marker point to be monitored from the two-dimensional image.

[0067] After the industrial camera is used to photograph the marker point to be monitored, the method of the present invention further comprises:

[0068] Adjust the position of the industrial camera until the marker point to be monitored is located in the center of the industrial camera's field of view.

[0069] Through the above technical solution, the distribution of multiple monitored marker points can be preliminarily captured by using an industrial camera to shoot the marker points to be monitored. If it is identified that multiple monitored marker points are offset from the field of view of the industrial camera, the position and posture of the industrial camera can be corrected through intelligent gimbal equipment or manually until the marker points to be monitored are located in the center of the industrial camera's field of view, thereby improving the monitoring accuracy.

[0070] Specifically, the above step S20 includes:

[0071] S21: Use an industrial camera to photograph the marker points to be monitored, and store the photographed two-dimensional images in an algorithm processing base station.

[0072] S22: The built-in target detection algorithm of the algorithm processing base station is used to select the light spot of the two-dimensional image, and the pixel coordinates of each light spot in the two-dimensional image and the ROI (Region of Interest) of the light spot distribution are obtained.

[0073] S23: Calculate the centroid pixel coordinates and numerical radius of each light spot according to the ROI of each light spot distribution and the pixel coordinates of each light spot in the two-dimensional image.

[0074] In this embodiment, the specific calculation formula of step S23 is:

[0075]

[0076] Among them, S k is the image pixel area of ​​the kth light spot, u p,k is the pixel horizontal coordinate of the center of mass of the kth spot in the two-dimensional coordinate system, v p,k is the centroid pixel ordinate of the kth spot centroid in the two-dimensional coordinate system, H k (i, j) is the two-dimensional pixel coordinate of the kth light spot in the two-dimensional coordinate system, i is the two-dimensional horizontal coordinate value in the two-dimensional coordinate system, j is the two-dimensional vertical coordinate value in the two-dimensional coordinate system, and is the numerical radius of the kth light spot.

[0077] S24: Using the centroid pixel coordinates as the positioning coordinates of the marker point to be monitored, and using the numerical radius as the pixel radius of the marker point to be monitored.

[0078] Through the above steps S21-S24, the two-dimensional positioning information of each marker point to be monitored in the two-dimensional image can be calculated.

[0079] It should be additionally explained that, after step S22, the method of the present invention further includes:

[0080] S201: Calculating a light spot threshold of each light spot based on an image pixel area of ​​each light spot in the two-dimensional image;

[0081] S202: Segment the light spot whose ROI is greater than or equal to the corresponding light spot threshold into the light spot actual area.

[0082] Specifically, the calculation formula of step S201 is:

[0083]

[0084] thr k =A k +ind k (6)

[0085] Among them, A k is the image pixel mean of the kth spot, thrk is the corresponding spot threshold of the kth spot. k is the image statistic of the kth light spot, which can be the mode or standard deviation, and is selected according to the specific tunnel scene and the corresponding target detection effect.

[0086] Through the above steps S201-S202, the method of the present invention can automatically calculate the spot threshold and segment the actual spot area according to the spot threshold, has a strong threshold adaptive ability, can better adapt to the harsh environment in the tunnel, and maximize the accuracy of identifying blurred and deformed spots in harsh tunnel environments such as insufficient light, temperature, and distance changes.

[0087] S30: Using a total station to mark the marker points to be monitored, obtaining the point data of the marker points to be monitored, and using the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera.

[0088] In this embodiment, a total station is used to Figure 2 The marker points monitored by the two industrial cameras are marked respectively. The point data is the three-dimensional physical coordinates of the marker point to be monitored in the three-dimensional coordinate system, and the three-dimensional coordinates of the industrial camera are the three-dimensional physical coordinates of the industrial camera in the three-dimensional coordinate system. The specific formula for calculating the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera in step S30 using the Euclidean distance formula is:

[0089]

[0090] Among them, X ii is the X-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system, ii is the Y-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system, and Z ii is the Z-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system; jj is the X-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, Y jj is the Y-axis coordinate of the jj-th industrial camera in the three-dimensional coordinate system, Z jj is the Z-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system; Dis ii,jj is the physical size distance from the ii-th marker point to be monitored to the jj-th industrial camera, which serves as the three-dimensional positioning information.

[0091] S40: Match the two-dimensional positioning information with the three-dimensional positioning information to obtain a two-dimensional-three-dimensional positioning information pair for the same marker point to be monitored.

[0092] Specifically, step S40 includes:

[0093] S41: Calculate the pixel magnification of the two-dimensional image;

[0094] S42: Based on the pixel magnification factor, a pairing relationship model between the pixel radius and the physical size distance of the marker point to be monitored is established;

[0095] S43: randomly placing multiple two-dimensional positioning information and multiple three-dimensional positioning information into a pairing relationship model, matching the information of multiple markers to be monitored in a two-dimensional coordinate system and a three-dimensional coordinate system, and obtaining multiple two-dimensional-three-dimensional positioning information pairs.

[0096] It should be explained that the above steps S41-S43 are used to utilize the inherent rules of the physical size distance between the monitored marker point and the industrial camera and the pixel radius of the monitored marker point in the image to achieve a one-to-one correspondence between the two-dimensional positioning information and the three-dimensional positioning information, thereby automatically constructing a two-dimensional-three-dimensional positioning information matching pair of the marker point.

[0097] It needs further explanation that Figure 3 As shown in the figure, the circular structure of the tunnel and the layout of the marker points to be monitored are regular. The layout of the marker points to be monitored is more regular in the industrial camera imaging image. The size of the marker point in the image is directly related to the straight-line distance from the industrial camera to the marker point. The corresponding proportional relationship is as follows: Figure 4 As shown, using the above rules, steps S41-S43 can be obtained. Among them, Figure 4 The proportional relationship formula shown is specifically:

[0098]

[0099] Among them, H1 and H2 are the actual physical sizes, d1 and d2 are the physical sizes of the image, f is the focal length of the camera, and L k,1 is the vertical distance from the optical center of the kth industrial camera to the tunnel section where the marker point to be monitored is located. According to the proportional relationship, the equation relationship can be listed as follows:

[0100]

[0101] From the above proportional relationship (8), we can get that the actual physical size ΔH and the image physical size Δd only differ by f and L. k,1 The related fixed proportional coefficient, ΔH, is the diameter of the mark point to be monitored. This diameter is fixed and known, so it can be obtained that the imaging physical size of the mark point to be monitored in the image within the same section is also fixed.

[0102] Based on the proportional relationships (8) and (9), more specifically, the specific formula of step S41 is:

[0103]

[0104] Where d is the pixel size of the industrial camera, L k+1,1 K is the vertical distance from the optical center of the k+1th industrial camera to the tunnel section where the marker to be monitored is located. k+1 is the pixel magnification of the k+1th industrial camera 2D image (unit: mm / pixel).

[0105] Taking into account the measurement error, the pairing relationship model of step S42 is specifically:

[0106]

[0107]

[0108] Wherein, formula (13) represents the correspondence between the pixel radius in the two-dimensional positioning information of the marker point to be monitored and the physical size distance D of the three-dimensional positioning information of the marker point to be monitored, and formulas (11) and (12) are the constraints of formula (13).

[0109] It should be explained that the pixel size and focal length of the industrial camera are constant for different markers to be monitored. Therefore, the main constraint affecting formula (11) is the vertical distance L from the optical center of the industrial camera to the tunnel section where the marker to be monitored is located. k+1,1 , so as to distinguish the imaging spots by using the tunnel depth information as the judging standard. Generally, the longitudinal field of view of each industrial camera is between 10m and 200m, so the three-dimensional positioning information Dis of the industrial camera in the three-dimensional coordinate system ii,jj The tunnel section radius R' must satisfy the constraint formula (12). Multiple two-dimensional positioning information and multiple three-dimensional positioning information are randomly placed in the pairing relationship model. If a pair of two-dimensional positioning information and three-dimensional positioning information satisfies the above relationship formulas (11)-(13), it means that the pair of two-dimensional positioning information and three-dimensional positioning information match each other and constitute a two-dimensional-three-dimensional positioning information pair. Such two-dimensional-three-dimensional positioning information pair can be stored in a database linked list structure.

[0110] Preferably, before the above step S43, the method of the present invention further includes:

[0111] S400: screening out multiple groups of marker points to be monitored located on the same section according to the pixel radius of the multiple light spots;

[0112] S401: When placing a plurality of two-dimensional positioning information and a plurality of three-dimensional positioning information into the pairing relationship model, it is limited that the number of marker points to be monitored located in the same section is less than or equal to 2.

[0113] Compared with the prior art, the method of the present invention can screen the marker points to be monitored in different tunnel sections, limit the number of marker points to be monitored in the same section to be less than or equal to 2, avoid singularity in model solution, and overcome the influence of different depth sections on two-dimensional image monitoring using industrial cameras.

[0114] Preferably, after step S400, the method of the present invention further comprises:

[0115] Based on multiple groups of marker points to be monitored located in the same section, the curvature of the half-section of the tunnel is approximated to 0.

[0116] Specifically, Figure 5 As shown in the figure, the marker points to be monitored in a tunnel section can be approximately regarded as having the same camera tunnel depth information. Under the premise of satisfying formula (12), the half-segment of the pipe section monitored by the industrial camera can be approximately regarded as a straight line segment with a curvature close to 0. The straight line direction of the half-segment of the pipe section is the direction of the line connecting the geometric center of the head section and the geometric center of the end section of the pipe section.

[0117] Through the above technical scheme, the method of the present invention combines the specific longitudinal field of view of the industrial camera and the curvature of the tunnel turning point (half-section pipe section). When analyzing the half-section pipe section, the pipe section is straight instead of curved, so as to facilitate the derivation of the distribution law of the marking points of different sections, and facilitate the distinction of the marking points in combination with the fixed setting law of the marking points of each section, thereby theoretically reducing the false detection rate of the light spot.

[0118] After the above step S40, the method of the present invention further includes:

[0119] S50: According to the pairing of the two-dimensional and three-dimensional positioning information, the posture of the industrial camera is globally solved to obtain the posture parameters.

[0120] In this embodiment, it is assumed that the rotation matrix of the marker to be monitored, which transforms the three-dimensional physical coordinates in the three-dimensional coordinate system into the two-dimensional pixel coordinates in the two-dimensional coordinate system, is R", and the translation vector is T", then their expressions are:

[0121]

[0122] According to the central projection principle, the two-dimensional pixel coordinates and three-dimensional physical coordinates of each marker point to be monitored can be listed as a set of equations:

[0123]

[0124] Among them, C x and C y is the principal point of the two-dimensional image, that is, the two-dimensional pixel coordinate of the intersection of the optical axis and the image plane, F x and F yis the equivalent focal length in the x-direction and y-direction in the two-dimensional coordinate system. The two-dimensional-three-dimensional positioning information matching pair of each marker to be monitored can be listed as a set of equations as shown in formula (15). There are 12 parameters involved in the operation of formulas (14)-(15), the internal parameters of the camera, namely C x , C y 、F x 、F y are known parameters, and the parameters r''0-r''8 in the rotation matrix and the translation vector T'' x 、T'' y 、T'' z is the value to be solved. To solve the above industrial camera’s posture, combined with the intrinsic quantitative relationship between the parameters of R” and T”, it is necessary to substitute 6 or more sets of 2D-3D positioning information matching pairs of the landmarks to be monitored into formulas (14)-(15) to obtain a unique solution.

[0125] S60: Back-projecting the three-dimensional positioning information onto the two-dimensional image, and calculating a back-projection error.

[0126] Specifically, the calculation formula of the above step S60 is:

[0127]

[0128] Where N is the total number of light spots in the two-dimensional image; u′ p,k is the two-dimensional horizontal coordinate of the kth marker point to be monitored, which is back-projected from the three-dimensional positioning information to the two-dimensional image, v′ p,k It is the two-dimensional ordinate of the kth marker point to be monitored, which is back-projected from the three-dimensional positioning information to the two-dimensional image.

[0129] S70: Determine whether the back-projection error is less than the error threshold. If so, based on the posture parameters, convert the two-dimensional positioning information into three-dimensional physical coordinates in the three-dimensional coordinate system, and perform two-dimensional-three-dimensional joint calibration on the marker points to be monitored that have completed the three-dimensional conversion.

[0130] Specifically, if the back-projection error is less than the error threshold, under the premise that the half-segment of the pipe section monitored by the industrial camera is approximately regarded as a straight line segment with a curvature close to 0, it means that the imaging spot coordinates of the marker point to be monitored in the two industrial camera images can be unified, and it is assumed that the change along the extension direction of the tunnel pipe section is small and negligible. According to the rotation matrix and translation vector from the two industrial cameras to the total station obtained by formula (14) and formula (15), the conversion formula between the two-dimensional pixel coordinates and the three-dimensional physical coordinates of the marker point to be monitored ignoring the change along the extension direction of the tunnel pipe section can be obtained:

[0131]

[0132] The numbers 1 and 2 in the subscripts of formula (17) represent industrial cameras 1 and 2 that monitor the same landmark. The formula for converting the two-dimensional image coordinates obtained by the two industrial cameras into three-dimensional physical coordinates in the total station coordinate system is as follows:

[0133]

[0134] Through formula (18), the joint calibration of the joint monitoring system can be achieved.

[0135] It should be noted that when the marker point to be monitored is displaced, the expression for calculating the marker point to be monitored in the total station coordinate system is:

[0136]

[0137] Among them, (X′ k,1 ,Y′ k,1 ,1) and (X′ k,2 ,Y′ k,2 ,1) is the coordinate of the same marker point to be monitored by two industrial cameras in the three-dimensional coordinate system after displacement, Δx is the X-axis offset of the marker point to be monitored in the two-dimensional image, and Δy is the Y-axis offset of the marker point to be monitored in the two-dimensional image.

[0138] Through the above steps S10-S70, compared with the prior art, the method of the present invention unifies the marker points to be monitored by the industrial camera and the total station in the non-public field of view in the coordinate system, and performs a two-dimensional-three-dimensional joint calibration on the marker points to be monitored, which combines the advantages of the total station and the industrial camera for tunnel monitoring, and overcomes the monitoring defects of the total station or the industrial camera alone. In addition, before solving the posture of the industrial camera, the method of the present invention fully utilizes the feature that there are enough marker points to be monitored in the tunnel scene to perform two-dimensional-three-dimensional matching calculations at multiple points, reducing the situation where the industrial camera posture is multiple solutions due to the lack of matching points, and improving the robustness and accuracy of the industrial camera calibration.

[0139] Preferably, after step S70, the method of the present invention further comprises:

[0140] If the back-projection error is greater than or equal to the error threshold, the RANSAC algorithm is used to filter out the mismatched two-dimensional positioning information and three-dimensional positioning information, and the process returns to step S50.

[0141] Furthermore, the method of the present invention uses the RANSAC framework to filter out points with low matching accuracy or mismatch one by one, and lists the filtered point set in the equation set for solving until the reprojection error is less than the set threshold or the number of iterations exceeds the set iteration limit, and the RANSAC algorithm is terminated.

[0142] Compared with the prior art, the method of the present invention can screen out the marker points to be detected that are mismatched or have unsatisfactory matching effects, thereby improving the accuracy of the two-dimensional-three-dimensional joint calibration of the method of the present invention.

[0143] Embodiment 2

[0144] like Figure 6 As shown, based on the first embodiment, this embodiment discloses an automatic calibration system based on joint point finding of a total station and a camera, the system comprising:

[0145] The coordinate establishment module is used to establish a two-dimensional coordinate system with the industrial camera as the origin, and to establish a three-dimensional coordinate system with the total station as the origin;

[0146] A two-dimensional positioning information acquisition module is used to use an industrial camera to shoot the marker point to be monitored to obtain a two-dimensional image, and extract the two-dimensional positioning information of the marker point to be monitored from the two-dimensional image;

[0147] The three-dimensional positioning information acquisition module is used to use a total station to mark the marker points to be monitored, obtain the point data of the marker points to be monitored, and use the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera;

[0148] An information pairing module is used to match the two-dimensional positioning information with the three-dimensional positioning information to obtain a two-dimensional-three-dimensional positioning information pairing for the same marker point to be monitored;

[0149] The posture acquisition module is used to globally solve the posture of the industrial camera based on the pairing of 2D-3D positioning information to obtain the posture parameters;

[0150] An error calculation module is used to back-project the three-dimensional positioning information onto a two-dimensional image and calculate a back-projection error;

[0151] The joint calibration module is used to determine whether the back-projection error is less than the error threshold. If so, based on the posture parameters, the two-dimensional positioning information is converted into three-dimensional physical coordinates in the three-dimensional coordinate system, and the two-dimensional-three-dimensional joint calibration is performed on the monitored landmarks that have completed the three-dimensional conversion.

[0152] Those skilled in the art can understand that all or part of the steps of implementing the above-mentioned embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, etc. Various media that can store program codes.

[0153] For those skilled in the art, various other corresponding changes and deformations can be made according to the technical solutions and concepts described above, and all of these changes and deformations should fall within the protection scope of the claims of the present invention.

Claims

1. An automatic calibration method based on joint point finding of total station and camera, characterized in that: include: S10: Establish a two-dimensional coordinate system with the industrial camera as the origin, and establish a three-dimensional coordinate system with the total station as the origin; S20: photographing the marker point to be monitored using the industrial camera to obtain a two-dimensional image, and extracting two-dimensional positioning information of the marker point to be monitored from the two-dimensional image; S30: using the total station to mark the marker points to be monitored, obtaining point data of the marker points to be monitored, and using the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera; S40: Matching the two-dimensional positioning information with the three-dimensional positioning information to obtain a two-dimensional-three-dimensional positioning information pair for the same marker point to be monitored; S50: performing a global solution for the posture of the industrial camera according to the pairing of the two-dimensional-three-dimensional positioning information to obtain posture parameters; S60: Back-projecting the three-dimensional positioning information onto the two-dimensional image, and calculating a back-projection error; S70: Determine whether the back-projection error is less than an error threshold. If so, based on the posture parameters, convert the two-dimensional positioning information into three-dimensional physical coordinates in the three-dimensional coordinate system, and perform two-dimensional-three-dimensional joint calibration on the marker points to be monitored that have completed the three-dimensional conversion.

2. The automatic calibration method based on joint point finding of total station and camera according to claim 1, characterized in that: After step S70, the method further includes: If the back-projection error is greater than or equal to the error threshold, the RANSAC algorithm is used to filter out the mismatched two-dimensional positioning information and three-dimensional positioning information, and the process returns to step S50.

3. The automatic calibration method based on joint point finding of total station and camera according to claim 1, characterized in that: The two-dimensional positioning information at least includes the positioning coordinates of the marker point to be monitored and the pixel radius of the marker point to be monitored, and the step S20 includes: S21: photographing the marker points to be monitored using the industrial camera, and storing the photographed two-dimensional images in an algorithm processing base station; S22: Using the built-in target detection algorithm of the algorithm processing base station to frame the light spot of the two-dimensional image, and obtain the pixel coordinates of each light spot in the two-dimensional image and the ROI of the light spot distribution; S23: Calculating the centroid pixel coordinates and numerical radius of each light spot according to the ROI of each light spot distribution and the pixel coordinates of each light spot in the two-dimensional image; S24: Using the centroid pixel coordinates as the positioning coordinates of the marker point to be monitored, and using the numerical radius as the pixel radius of the marker point to be monitored.

4. The automatic calibration method based on joint point finding of total station and camera according to claim 3 is characterized in that: After step S22, the method further includes: S201: calculating a light spot threshold of each light spot based on an image pixel area of ​​each light spot in the two-dimensional image; S202: Segment the light spot whose ROI is greater than or equal to the corresponding light spot threshold into the light spot actual area.

5. The automatic calibration method based on joint point finding of total station and camera according to claim 3, characterized in that: The three-dimensional positioning information includes the physical size distance from the marker point to be monitored to the industrial camera, and the step S40 includes: S41: Calculating the pixel magnification of the two-dimensional image; S42: Based on the pixel magnification factor, a pairing relationship model between the pixel radius of the marker point to be monitored and the physical size distance is established; S43: randomly placing a plurality of two-dimensional positioning information and a plurality of three-dimensional positioning information into the pairing relationship model, matching the information of a plurality of marker points to be monitored in a two-dimensional coordinate system and a three-dimensional coordinate system, and obtaining a plurality of two-dimensional-three-dimensional positioning information pairs.

6. The automatic calibration method based on joint point finding of total station and camera according to claim 5, characterized in that: Before step S43, the method further includes: S400: screening out multiple groups of marker points to be monitored located on the same section according to the pixel radius of the multiple light spots; S401: When placing a plurality of two-dimensional positioning information and a plurality of three-dimensional positioning information into the pairing relationship model, it is limited that the number of marker points to be monitored located in the same section is less than or equal to 2.

7. The automatic calibration method based on joint point finding of total station and camera according to claim 6, characterized in that: After step S400, the method further includes: Based on multiple groups of marker points to be monitored located in the same section, the curvature of the half-section of the tunnel is approximated to 0.

8. The automatic calibration method based on joint point finding of total station and camera according to claim 1, characterized in that: The point data is the three-dimensional physical coordinates of the marker point to be monitored in the three-dimensional coordinate system, and the three-dimensional coordinates of the industrial camera are the three-dimensional physical coordinates of the industrial camera in the three-dimensional coordinate system. The specific formula for calculating the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera using the Euclidean distance formula in step S30 is: Among them, X ii is the X-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system, and Y ii is the Y-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system, and Z ii is the Z-axis coordinate of the ii-th marker point to be monitored in the three-dimensional coordinate system; jj is the X-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, and Y jj is the Y-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system, Z jj is the Z-axis coordinate of the jjth industrial camera in the three-dimensional coordinate system; Dis ii,jj The physical size distance from the ii-th marker point to be monitored to the jj-th industrial camera is used as the three-dimensional positioning information.

9. The automatic calibration method based on joint point finding of total station and camera according to claim 1, characterized in that: After the industrial camera is used to photograph the marker point to be monitored, the method further includes: The position and posture of the industrial camera is adjusted until the marker point to be monitored is located in the center of the field of view of the industrial camera.

10. An automatic calibration system based on joint point finding of total station and camera, characterized in that: The automatic calibration method based on joint point finding of a total station and a camera according to any one of claims 1 to 9 is adopted, and the system comprises: The coordinate establishment module is used to establish a two-dimensional coordinate system with the industrial camera as the origin, and to establish a three-dimensional coordinate system with the total station as the origin; A two-dimensional positioning information acquisition module is used to use the industrial camera to shoot the marker point to be monitored to obtain a two-dimensional image, and extract the two-dimensional positioning information of the marker point to be monitored from the two-dimensional image; A three-dimensional positioning information acquisition module is used to use the total station to mark the marker points to be monitored, obtain the point data of the marker points to be monitored, and use the Euclidean distance formula to calculate the three-dimensional positioning information of the point data relative to the three-dimensional coordinates of the industrial camera; An information pairing module, used to match the two-dimensional positioning information with the three-dimensional positioning information to obtain a two-dimensional-three-dimensional positioning information pairing for the same marker point to be monitored; A posture acquisition module, used to globally solve the posture of the industrial camera according to the two-dimensional-three-dimensional positioning information pairing to obtain posture parameters; An error calculation module, used for back-projecting the three-dimensional positioning information onto the two-dimensional image to calculate a back-projection error; The joint calibration module is used to determine whether the back-projection error is less than an error threshold. If so, based on the posture parameters, the two-dimensional positioning information is converted into three-dimensional physical coordinates in the three-dimensional coordinate system, and a two-dimensional-three-dimensional joint calibration is performed on the monitored marker points that have completed the three-dimensional conversion.

Images