Multi-sensor fusion detecting, positioning and guiding method for mounting tunnel davit

Abstract

The invention discloses a multi-sensor fusion detecting, positioning and guiding method for tunnel davit installation, and belongs to the technical field of tunnel engineering construction.The method comprises the steps that a lifting module is controlled to move a detection module comprising a camera assembly and a laser sensor, and images of upper and lower screw holes are sequentially collected and processed; obtaining center pixel coordinates, intervals and flange face inclination angles of the screw holes; scanning and collecting point clouds by using a laser radar, matching the point clouds with the tunnel feature map after filtering, downsampling and the like, and determining the accurate point location of the channel bolt; coordinates of the mechanical arm and the camera are calibrated, the channel offset is obtained, and automatic mounting of the davit is achieved in combination with screw hole data. Accurate detection, positioning and guiding in the tunnel davit mounting process are achieved, the mounting precision and efficiency are improved, and the construction quality and safety are guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of tunnel engineering construction, and in particular to a multi-sensor fusion detection, positioning and guiding method for tunnel hanger column installation. BACKGROUND

[0002] In the technical field of tunnel engineering construction, the installation quality of the tunnel hanger column plays a key role in the stability and safety of the catenary system, and its accurate installation is an important foundation for ensuring power transmission and safe train operation in the tunnel.

[0003] Currently, there are many problems in the detection, positioning and guiding methods during the installation of the tunnel hanger column. Traditional methods rely mostly on manual measurement and experience-based judgment. When manual measurement tools are used to measure the installation position of the hanger column, it is difficult to accurately obtain key parameters such as screw hole position, spacing and flange angle due to the limitations of measurement tool accuracy and human operation errors, resulting in low installation accuracy of the hanger column, which is prone to deviation and tilting, and affects the normal operation of the catenary system. Some automatic installation methods use single sensors, such as laser range finders or cameras for detection and positioning. Due to the limitations of single sensors, they cannot obtain multi-dimensional information required for installation. In complex tunnel environments, they are difficult to achieve accurate detection and positioning in the face of factors such as light changes and tunnel structure diversity. Moreover, once the sensor fails, the entire installation process will be severely affected, lacking reliability and stability. In addition, existing methods also have problems such as poor data synchronization, low processing efficiency, and unreasonable trajectory planning of the mechanical arm in data processing and mechanical arm control, resulting in low installation efficiency and failing to meet the needs of modern tunnel engineering efficient construction.

[0004] Therefore, there is an urgent need for a detection, positioning and guiding method for tunnel hanger column installation that can overcome the above-mentioned defects and achieve high precision, high efficiency and high reliability. SUMMARY

[0005] Therefore, the present application provides a multi-sensor fusion detection, positioning and guiding method for tunnel hanger column installation, which at least partially solves the problems in the prior art.

[0006] The present application provides a multi-sensor fusion detection, positioning and guiding method for tunnel hanger column installation, comprising the following steps:

[0007] S1: The system controls the lifting module to move the detection module containing two camera assemblies and a laser sensor to an upper detection position, the camera assemblies collect left and right screw hole images, the screw hole shapes are judged through threshold binarization, contour extraction and template matching, the screw hole center pixel coordinates (X1, Y1), (X2, Y2) are located, the laser sensor measures the distance, when the difference between the two measurement values is less than a set value, the average value L1 of the distance is calculated, the magnification N1 is calculated through the L1 lookup table or polynomial, and the upper screw hole spacing D1 is calculated;

[0008] S2: Based on the upper screw hole spacing D1 obtained in S1, the detection module is controlled to be vertically lowered by a distance H to a lower detection position, the camera assemblies collect left and right screw hole images and repeat the image processing procedure of S1, the screw hole center pixel coordinates (X3, Y3), (X4, Y4) are located, the laser sensor measures the distance and calculates the average value L2, the lower screw hole spacing D2 is calculated, and the flange surface inclination angle a is calculated according to the difference between L1 and L2.

[0009] S3: Based on the flange surface inclination angle a obtained in S2, the laser radar scans in a Z-shaped path, collects three-dimensional point clouds of the channel, the reflective sticker and the tunnel, removes outliers through statistical filtering, judges the ground points by the ray method ground filtering, reduces and samples the data by voxel, and converts the point clouds to the carrier coordinate system.

[0010] S4: According to the converted point cloud data in S3, the current frame point cloud is matched with the tunnel feature map, the tunnel center line and the channel spatial straight line model are fitted, the target area point cloud is extracted, the normal vector estimation, RANSAC plane fitting and Euclidean clustering segmentation are finely processed, the transformation matrix is solved based on SVD decomposition, and the accurate point position and size of the channel bolt are determined.

[0011] S5: On the premise that the accurate point position and size of the channel bolt are determined in S4, the origin of the tool coordinate system of the mechanical arm is calibrated by the three-point method, after the 2D area array camera and the mechanical arm coordinate are calibrated, the channel image is collected and processed through binarization, contour filling and minimum circumscribed rectangle calculation, the channel angle deflection and Y-direction offset are obtained, the mechanical arm is moved according to the offset and aligned with the channel, and the column automatic installation is performed combined with the upper screw hole spacing D1, the lower screw hole spacing D2 and the flange surface inclination angle a obtained in S1 and S2.

[0012] According to a specific implementation manner of the embodiment of the application, in step S1, when the camera assembly collects the image, the exposure time is adaptively adjusted according to the light intensity in the tunnel, the light intensity is obtained through the ambient light sensor installed on the detection module, the exposure time t and the light intensity I satisfy the function relationship t=k / I, k is a preset constant, so as to ensure that the collected screw hole image is clear and improve the accuracy of the screw hole center pixel coordinate positioning.

[0013] According to a specific implementation manner of the embodiment of the present application, in step S2, during the laser radar scanning process, the scanning frequency is dynamically adjusted according to the detection module moving speed, when the detection module moving speed v is greater than a set threshold v0, the scanning frequency f is increased to n times of the original frequency, to ensure that the point cloud data density collected during the detection module moving process meets the channel positioning and measurement accuracy requirement based on the flange surface inclination angle a, and the set threshold v0 and the multiple n are preset.

[0014] According to a specific implementation manner of the embodiment of the present application, in step S3, the 2D area array camera lens adopts a fisheye lens, the distortion parameters of the fisheye lens are accurately obtained in the calibration process and used for subsequent image processing, the collected channel image is de-distorted by a distortion correction algorithm, so that the channel edge profile is more accurate, and the angle deflection and Y-direction offset calculation accuracy based on the channel bolt accurate point position and size calculation is improved.

[0015] According to a specific implementation manner of the embodiment of the present application, in steps S1 and S2, the sensor data collection is realized in hardware synchronization, the data of each sensor is collected by triggering the same clock source, to ensure that when the upper hole spacing D1, the lower hole spacing D2 and the flange surface inclination angle a and other data are obtained, the consistency of the davit flange hole detection data and the channel point cloud data in time is ensured, and the positioning error caused by different time is reduced.

[0016] According to a specific implementation manner of the embodiment of the present application, in step S2, when the point cloud data collected by the laser radar is de-noised, a bilateral filtering algorithm is adopted, which can remove noise while retaining point cloud edge information, and the spatial distance weight function and the gray similarity weight function of the bilateral filtering are dynamically adjusted according to the point cloud density and noise characteristics, to achieve the best de-noising effect and ensure the accuracy of the point cloud data for calculating the flange surface inclination angle a.

[0017] According to a specific implementation manner of the embodiment of the present application, in step S5, the trajectory planning algorithm based on model predictive control (MPC) is adopted in the mechanical arm walking process, the calculated channel angle deflection and Y-direction offset are combined with the mechanical arm dynamics model, the future states of the mechanical arm at multiple time points are predicted, the control input of the mechanical arm is optimized at each sampling time, and the upper hole spacing D1, the lower hole spacing D2 and the flange surface inclination angle a are combined, so that the mechanical arm can quickly and smoothly align the channel and complete the davit installation, and the collision risk is avoided.

[0018] According to a specific implementation manner of the embodiment of the present application, when a sensor fails, the system automatically switches to the redundant sensor working mode, to ensure the continuity of the detection, positioning and guidance during the acquisition of the upper hole spacing D1, the lower hole spacing D2, the flange surface inclination angle a, the channel bolt accurate point position and size and other key data.

[0019] According to a specific implementation manner of the embodiment of the application, in step S1, the range of the distance measured by the laser sensor is adjusted according to the size of the hanger post flange and the installation distance of the detection module and the flange, so that the laser sensor can accurately measure the distance of the upper and lower detection positions, and the measurement accuracy is better than ±0.5 mm, meeting the high-precision requirements of accurately calculating the upper hole spacing D1 and subsequently calculating the flange surface inclination angle a based on the difference between L1 and L2.

[0020] According to a specific implementation manner of the embodiment of the application, in step S2, the tunnel feature map is pre-constructed according to different tunnel types and previous engineering data, and the feature map contains information such as tunnel profile and channel distribution. In the point cloud matching process, the corresponding feature map is selected according to the current tunnel type for matching, so as to improve the efficiency and accuracy of channel positioning based on the flange surface inclination angle a and other data.

[0021] The application scheme has the following beneficial effects:

[0022] High-precision detection and positioning: Through the fusion of multiple sensors such as camera assemblies, laser sensors and laser radars, multi-dimensional information such as pixel coordinates of screw hole centers, spacing, flange surface inclination angle and accurate points of channel bolts can be accurately obtained. Compared with traditional manual measurement or single sensor detection, human operation errors and single sensor functional limitations are effectively avoided, and the system can work stably in complex tunnel environments, greatly improving the positioning accuracy of hanger post installation and ensuring the stability and safety of the overhead line system after the hanger post installation.

[0023] Adaptive and dynamic adjustment: The exposure time of the camera assembly is adaptively adjusted according to the light intensity in the tunnel, and the scanning frequency of the laser radar dynamically changes with the moving speed of the detection module, so that the system can adapt to different light conditions and changes in the detection environment. At the same time, the 2D area camera uses a fisheye lens and performs distortion correction, improving the accuracy of image acquisition and processing and further ensuring the reliability of detection and positioning, solving the problem of poor detection effect of existing methods in complex environments.

[0024] Efficient data processing and control: The data acquisition of each sensor is synchronized in hardware, reducing the positioning error caused by different time synchronization; the bilateral filtering algorithm is used to denoise the laser radar point cloud data, retaining the edge information; the trajectory planning algorithm based on model predictive control (MPC) is used for mechanical arm walking, and the control input is optimized combined with the dynamics model. These measures significantly improve the data processing efficiency and mechanical arm control accuracy, making the hanger post installation process faster and smoother, effectively improving the construction efficiency and meeting the demand of modern tunnel engineering for efficient construction.

[0025] High reliability and stability: when a sensor fails, the system automatically switches to a redundant sensor mode, ensuring that the installation process is not affected, greatly enhancing the reliability and stability of the system. Compared with the existing single sensor automatic installation method, the present application can still ensure the smooth progress of the davit installation work when facing sudden situations such as sensor failure, reducing the construction risk. BRIEF DESCRIPTION OF DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0027] Figure 1 The flange screw hole detection and inclination measurement diagram provided for the embodiments of the present application;

[0028] Figure 2 The overall operation method flowchart provided for the embodiments of the present application;

[0029] Figure 3 The current frame point cloud matching and positioning schematic diagram provided for the embodiments of the present application;

[0030] Figure 4 The mechanical hand three-point calibration flowchart provided for the embodiments of the present application;

[0031] Figure 5 The flowchart of the visual detection module processing image provided for the embodiments of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application will be described in detail below with reference to the drawings.

[0033] Reference Figure 1 , Figure 2 , Figure 3 , Figure 4 The present application discloses a kind of detection, positioning, guiding method of multi-sensing fusion for tunnel davit installation, comprising the following steps:

[0034] S1: system control lifting module moves detection module containing two camera components and laser sensor to upper detection position, camera component gathers left and right screw hole image, judges screw hole shape by threshold binaryzation, contour extraction and template matching, locates screw hole center pixel coordinates (X1, Y1), (X2, Y2), laser sensor measures distance, when the difference between two measurement values is less than set value, calculate the average value L1 of ranging, look up table or polynomial calculation magnification N1 through L1, calculate the spacing D1 of upper screw hole;

[0035] S2: Based on the upper screw hole spacing D1 obtained in S1, the detection module is controlled to move vertically downward by a distance H to a lower detection position, the camera assembly collects left and right screw hole images, and the image processing procedure of S1 is repeated to locate the screw hole center pixel coordinates (X3, Y3) and (X4, Y4), the laser sensor measures the distance and calculates the average value L2, the lower screw hole spacing D2 is calculated, and the flange surface inclination angle a is calculated according to the difference between L1 and L2;

[0036] S3: Based on the flange surface inclination angle a obtained in S2, the laser radar scans in a zigzag path, collects three-dimensional point clouds of the channel, the reflective sticker and the tunnel, removes outliers through statistical filtering, determines ground points by ray ground filtering, reduces and samples data by voxel, and converts the point cloud to a carrier coordinate system;

[0037] S4: According to the converted point cloud data in S3, the current frame point cloud is matched with the tunnel feature map, the tunnel center line and the channel space straight line model are fitted, the target area point cloud is extracted, the normal vector estimation, RANSAC plane fitting and Euclidean clustering segmentation are used for fine processing, the transformation matrix is solved based on SVD decomposition, and the accurate point position and size of the channel bolt are determined;

[0038] S5: On the premise that the accurate point position and size of the channel bolt are determined in S4, the origin of the tool coordinate system of the mechanical arm is calibrated by the three-point method, after the 2D area array camera and the mechanical arm coordinate are calibrated, the channel image is collected, binarization, contour filling and minimum circumscribed rectangle calculation are performed, the channel angle deflection and Y direction offset are obtained, the mechanical arm moves according to the offset and aligns with the channel, and the flange surface inclination angle a, the upper screw hole spacing D1 and the lower screw hole spacing D2 obtained in S1 and S2 are combined to automatically install the suspension post.

[0039] Specifically, the present application discloses a multi-sensor fusion detection, positioning and guiding method suitable for automatic installation of catenary suspension posts in a pre-buried channel of a tunnel, which comprises suspension post flange screw hole detection and inclination angle measurement, channel positioning and suspension post installation guiding methods. The method can realize automatic and accurate installation of suspension posts in a pre-buried channel of a tunnel in combination with a catenary suspension post automatic installation device. The present application can effectively solve the problems of high labor intensity, low work efficiency, high work risk and poor working environment of tunnel suspension post installation workers. Based on the fact that most enterprises are in the critical period from manual production to automatic production, the present device has broad market prospects and high expected economic benefits.

[0040] The flange screw hole detection and inclination angle measurement are as shown in Figure 1 The detection procedure comprises:

[0041] (1) The system controls the lifting module to move, so that the detection module (two camera assemblies and two laser sensors) installed on the lifting module moves to an upper detection position.

[0042] (2) The system controls two camera assemblies to collect the left and right screw hole images of the current position, processes the images, and determines whether the screw hole shape is abnormal. If yes, the system alarms and prompts the operator to replace the hanger column. If no, the system locates the upper left and right screw hole center pixel coordinates (X1, Y1) and (X2, Y2).

[0043] Each camera assembly mainly consists of an area array camera, a fixed focus lens, and a 90-degree ring light source. When the assembly works, the 90-degree ring light source directly irradiates the object, so that the brightness around the hole edge of the collected image is significantly greater than the hole. The image processing procedure is as follows: 1. The collected gray-scale image is subjected to threshold binary processing; 2. All contour point sets, including closed and non-closed contour point sets, are extracted from the binary image; 3. All contour point sets are traversed to find whether there is a closed contour point set with a similarity to the template screw hole contour point set reaching a set value. If no, the screw hole shape is determined to be abnormal. If yes, the centroid of the contour point set is calculated as the screw hole center.

[0044] (3) The system controls two laser sensors to measure the distance of the current position, determines whether the difference between the two measured values is less than a set value, and if yes, it indicates that the hanger column flange is not tilted in the horizontal direction and meets the design requirements. The average value of the two measured values is obtained as the distance L1 of the distance measuring sensor from the flange when detecting the upper position. If no, the hanger column flange is determined to be unqualified, the system alarms, and prompts the operator to replace the hanger column.

[0045] (4) According to L1, the distance between the upper left and right screw holes is calculated, and it is determined whether the distance value is less than a set value. If no, the hanger column flange is determined to be unqualified, the system alarms, and prompts the operator to replace the hanger column.

[0046] L1 is used as the object distance of the two camera assemblies. For a fixed focus lens, the depth of field range is in focus, but the magnification N (the ratio of the physical distance of the image measurement object to the pixel distance) is different at different positions in the depth of field direction. Therefore, the magnification of the set object distance range is calibrated in advance, that is, within the set object distance range, different object distance values correspond to different magnification values. In this way, the corresponding magnification N1 can be obtained by looking up the table, or the current object distance value can be substituted into the polynomial of magnification and object distance derived from the corresponding magnification value of the object distance value to obtain the corresponding magnification N1.

[0047] The distance D1 between the upper left and right screw holes is calculated by the following formula.

[0048]

[0049] (5) The system controls the lifting module to move vertically downward by a distance H, so that the detection module moves to the lower detection position.

[0050] (6) The system controls two camera modules to collect the left and right screw hole images of the current position, processes the images, and determines whether the screw hole shape is abnormal. If so, the system alarms and prompts the operator to replace the hanger column. If not, the left and right screw hole center pixel coordinates (X3, Y3) and (X4, Y4) are positioned. The image processing procedure and steps (2) are the same and will not be repeated.

[0051] (7) The system controls two laser sensors to measure the distance of the current position, obtains the average value of the two measurement values, and uses it as the distance L2 of the ranging sensor from the flange when detecting the position below.

[0052] (8) According to L2, the distance between the left and right screw holes below is calculated, and it is determined whether the distance value is less than the set value. If not, the hanger column flange is determined to be unqualified, the system alarms, and prompts the operator to replace the hanger column.

[0053] The distance D2 between the left and right screw holes below is calculated by the following formula, which is the same as the calculation of D1 in step (4).

[0054]

[0055] Where N2 represents the magnification corresponding to the distance L2.

[0056] (9) Calculate the inclination angle a of the hanger column flange face. When L1>L2, When L1<L2,

[0057] (10) Calculate the distance between the left and right screw holes above and below, and determine whether both distances are less than the set value. If not, the hanger column flange is determined to be unqualified, the system alarms, and prompts the operator to replace the hanger column.

[0058] (11) Calculate the offset of the template screw hole center when the relative inclination angle a of the four screw hole centers is calculated. The template screw hole center is coaxial with the hanger column body.

[0059] The channel positioning and measurement process based on laser radar includes:

[0060] Collect the laser radar, preliminarily collect the channel shape and the point cloud of the reflective sticker, and the basic three-dimensional point cloud information of the entire tunnel and channel features, especially as basic feature template information.

[0061] 1. Laser radar data acquisition

[0062] During dynamic operation, the laser radar emits laser pulses according to the preset zigzag scanning path, receives the reflected signals, measures the distance and angle of each point, and generates real-time three-dimensional point cloud.

[0063] 2. Raw point cloud preprocessing

[0064] The original collected point cloud is preliminarily processed to remove noise and abnormal points. The point cloud is converted from the coordinate system of the laser radar itself to the carrier coordinate system.

[0065] The laser radar raw data needs to be parsed into Cartesian coordinates (x, y, z) and reflection intensity (I).

[0066]

[0067] Where d is the ranging value, θ is the horizontal angle, and φ is the pitch angle.

[0068] Denoising and filtering

[0069] Statistical filtering: remove outliers based on the distance distribution of neighboring points.

[0070] Calculate the average distance μi and standard deviation σi of the point pi to the nearest k points, and remove the points that satisfy || μi- μglobal|| > ασglobal (α is the threshold value, usually 1-3).

[0071] Ray ground filter (Ray Ground Filter)

[0072] Define the slope threshold Slocal (local slope) and Sgeneral (global slope), and the height threshold hth = r tan (s) (r is the horizontal distance of the point to the radar).

[0073] If the point height z ∈ [zground-hth, zground+hth], it is determined as a ground point.

[0074] Voxel down-sampling:

[0075] Divide the space into a voxel grid, and use the barycenter point to represent all points in the voxel 10:

[0076]

[0077] (N is the number of points in the voxel, which can reduce the data volume and retain the structural features).

[0078] 3. Current frame point cloud matching and positioning

[0079] The processed current frame point cloud is matched with a pre-established tunnel feature map, and if the matching is successful, secondary scanning is performed for accurate point cloud output. According to the tunnel point cloud data, the center line position of the arched top tunnel is fitted, and when perpendicular to the tunnel center line, the upper and lower single-channel information is determined, so as to obtain the included angle information a, b from c and the straight line starting point pi pj (i = 1, 2, 3; j = 1, 2, 3) of the tunnel center line, the upper channel and the lower channel, and a spatial straight line model L1 L2 L3 is established; the intersection point P1 of L1 and L2 is calculated, the intersection point P2 of L1 and L2 is calculated, and L3 = 100 mm is calculated, which is substituted into the straight line model tool to calculate the positions of the six bolts, as shown in Figure 3

[0080] 4. Target region point cloud extraction

[0081] Based on the positioning result, the current frame point cloud is transformed into the global feature map coordinate system. Then the surrounding range of the target point is drawn in the feature map. From the current frame point cloud transformed into the global coordinate system, the points falling within the ROI range are extracted.

[0082] 5. Target point cloud refinement and segmentation

[0083] In the target region point cloud extracted in step 4, further refinement is performed to ensure that only points belonging to the target itself are retained and the target can be segmented from the surrounding tunnel wall.

[0084] Normal vector and curvature calculation

[0085] Normal vector estimation (PCA)

[0086] Covariance matrix of the neighborhood of point pi:

[0087] Eigenvalue decomposition, the minimum eigenvalue corresponds to the normal vector direction.

[0088] Curvature calculation: (λ j Eigenvalue of C)

[0089] The greater the curvature, the more dramatic the surface change (such as edges).

[0090] Feature-based segmentation

[0091] RANSAC plane fitting (ground / building segmentation)

[0092] Model equation: ax + by + cz + d = 0, by iteratively optimizing inliers (point-to-plane distance less than threshold δ):

[0093]

[0094] ​Euclidean clustering segmentation (obstacle detection), based on KD-tree nearest neighbor search, merging clusters of points with distance less than a threshold ∈.

[0095] 6. Precise point determination

[0096] In the fine point cloud belonging to the target itself obtained in step 5, the precise position of the target is determined.

[0097] Objective: Minimize the distance between the source point cloud S and the target point cloud T.

[0098] Nearest point search:

[0099] For each si∈S, find the nearest point tj in T: j = argkmin||si-tk||, ||tk||.

[0100] Transformation matrix solving (SVD decomposition)

[0101] . Calculate the centroid

[0102] . Construct the covariance matrix

[0103] SVD decomposition Rotation matrix Translation vector t = tˉ - Rsˉ. Iterative update: repeat until convergence (change in transformation ||ΔR|| + ||Δt|| < ∈ or reach maximum number of iterations).

[0104] 7. Target size measurement

[0105] According to the fine point cloud belonging to the target itself obtained in step 5, the size of the target is calculated.

[0106] 8. Result output and application

[0107] Output the precise point position and size found for recording and visualization.

[0108] Mechanical hand three-point method calibration as shown in Figure 4 :

[0109]

[0110] Three-point method: the distance from the sphere center to any three points on the sphere surface is equal (radius R), forming a system of equations:

[0111] After eliminating the quadratic terms, it is transformed into a linear system of equations to solve the sphere center coordinates (x, y, z)

[0112] 1. Record the approach point 1 (initial pose):

[0113] Manually move the robot arm to accurately align the laser point with the target plate marker point.

[0114] Select tool coordinate system setup→3-point calibration→record current position as Approach Point 1 at the robot control interface.

[0115] 2. Record Approach Point 2 (rotate around tool Z axis):

[0116] Lift 50mm along the robot base coordinate system +Z direction (avoid collision).

[0117] Rotate 90°~180° around flange J6 axis (change pose but keep the laser spot hitting the same target point).

[0118] Record as Approach Point 2.

[0119] 3. Record Approach Point 3 (change pitch angle):

[0120] Lift the robot arm 50mm again.

[0121] Adjust J4 / J5 joints (pitch or yaw) to make the laser spot hit the target point again (pose is maximally different from the previous two times).

[0122] Record as Approach Point 3.

[0123] 4. Calculate TCP:

[0124] The robot system automatically calculates the tool coordinate system origin (TCP), i.e. the position of the laser spot center in the flange coordinate system [X_tcp, Y_tcp, Z_tcp].

[0125] 5. Verify calibration accuracy:

[0126] Control the robot to move to a new position and measure a target plate at a fixed distance (e.g. 300mm).

[0127] Compare the actual measured distance with the reference value by tape measure. If the error is >0.5mm, recalibrate.

[0128] Robot Y-direction vision alignment: Use the camera on the side of the device hanger installation module to accurately position the slot in the Y-direction (perpendicular to the slot direction), calculate the slot angle deflection and Y-direction offset

[0129] (1) Coordinate calibration of robot and 2D area array camera.

[0130] (2) According to the positioning information of the total station, the robot moves to the position directly below the hanger installation position.

[0131] (3) The robot main control system controls the distance sensor to measure the distance from the top of the tunnel. The nearby distance is set to the distance where the 2D area array is in focus and clear.

[0132] (4) The mechanical arm master control system communicates with the vision module, triggering the vision detection module to take pictures; the vision detection module processes the images, calculates the slot angle deflection and Y-direction pixel offset.

[0133] (5) Combined with the Y-direction pixel offset and the ranging sensor ranging value, the Y-direction distance offset is calculated.

[0134] (6) Determine whether the slot angle deflection and Y-direction distance offset are less than the set value. If so, the mechanical arm Y-direction visual alignment is completed. The mechanical arm moves according to the relative position of the four screw hole centers to the column body axis, the slot angle deflection and Y-direction distance offset, the relative position of the visual installation position and the center position of the column, and aligns the column with the slot for lifting and installation. If not, the mechanical arm moves according to the angle deflection and Y-direction distance offset. The mechanical arm master control system controls the ranging sensor to measure the distance from the top of the tunnel, and then executes steps (4) and (5).

[0135] The flow chart of processing images (detecting slots) by the vision detection module is shown in Figure 5 After obtaining the image of the slot from the 2D camera, first extract the slot area by gray scale binaryzation, calculate the connected domain to separate all closed contours, fill the holes in the closed contour, calculate the minimum circumscribed rectangle of the closed contour and the angle between the minimum circumscribed rectangle and the image coordinate system Y direction, select the contour area according to the length of the minimum circumscribed rectangle, and output the angle between the minimum circumscribed rectangle of the remaining contour after selection and the image coordinate system Y direction and the center point of the rectangle.

[0136] The details of each process include:

[0137] ① Apply binary threshold segmentation to extract dark areas.

[0138] ② Find all contours after binaryzation, calculate the connected domain to separate closed regions.

[0139] a. Scan the binary image row by row;

[0140] b. Neighborhood analysis (4 / 8 connectivity) is performed on the foreground pixels;

[0141] c. Use and find data structure to mark the connected domain;

[0142] d. Build contour tree structure (including hierarchical relationship).

[0143] The calculation formula is as follows:

[0144] C k = {(x, y) | binary(x, y) = 255 ∧ connected(x, y)}

[0145] Where, C k is the point set of the kth connected domain (contour), connected(x, y) is the connected relationship between the pixel points (4 / 8 neighborhood).

[0146] ③ Fill the holes inside the closed contour.

[0147] a. Create a full zero mask;

[0148] b. Traverse all contours;

[0149] c. Fill the outer contour without parent contour;

[0150] d. Fill the inner contour (hole) with parent contour.

[0151] The calculation formula is as follows:

[0152]

[0153] Where, outer is the area surrounded by the outer contour (without parent contour), inner is the area surrounded by the inner contour (hole), and filled(x, y).

[0154] ④ Calculate the minimum bounding rectangle of all closed contours.

[0155] Calculate the minimum bounding rectangle of the contour, calculate the minimum bounding rectangle center point coordinates, rectangle length and width, and rectangle angle. The minimum bounding rectangle calculation steps are:

[0156] a. Calculate the centroid:

[0157] Centroid is the average position of all points, which can be calculated by the following formula:

[0158]

[0159] Where, (x i ,y i ) is the coordinate of the ith point, and n is the total number of points.

[0160] b. Construct the covariance matrix (Covariance Matrix):

[0161] The covariance matrix is used to find the principal direction of the point set, and its formula is:

[0162]

[0163] Where, σ xx is the variance in the x direction, σ yy is the variance in the y direction, σ xy and σ yx are the covariances between x and y.

[0164] c. Calculate the eigenvalues and eigenvectors of the covariance matrix:

[0165] The eigenvalues and eigenvectors can be obtained by solving the following characteristic equation:

[0166]

[0167] This will give two eigenvalues λ1 and λ2, and the corresponding eigenvectors v1 and v2. The eigenvector corresponding to the larger eigenvalue represents the principal direction of the point set.

[0168] d. Determine the width and height of the minimum bounding rectangle:

[0169] Use the eigenvalues to determine the width and height of the minimum bounding rectangle:

[0170]

[0171] where w is the width of the rectangle, l is the length of the rectangle, is the square of the covariance between x and y coordinates.

[0172] e. Calculate the rotation angle in the Y direction:

[0173] The rotation angle in the Y direction θ y can be calculated by the slope of the eigenvector:

[0174] θ y = arctan2(v 1x , v 1y )

[0175] where v 1x is the component of the eigenvector v1 in the x direction, and v 1y is the component of the eigenvector v1 in the y direction.

[0176] ⑤ Screen the target area.

[0177] According to the length of the minimum bounding rectangle of the contour, retain the contour whose minimum bounding rectangle length is greater than the set length threshold, and filter out the contour that does not meet the threshold requirement.

[0178] ⑥ Output the result.

[0179] The screened contour is the target contour of the slot, and the center point (x, y) and the Y direction angle θ y of the output contour are output.

[0180] According to a specific implementation manner of the embodiment of the present application, in step S1, when the camera assembly collects images, the exposure time is adaptively adjusted according to the light intensity in the tunnel, the light intensity is obtained through the ambient light sensor installed on the detection module, the exposure time t and the light intensity I satisfy a function relationship t=k / I, k is a preset constant, so as to ensure that the collected screw hole image is clear and improve the accuracy of the positioning of the center pixel coordinates of the screw hole.

[0181] According to a specific implementation manner of the embodiment of the present application, in step S2, during the scanning of the laser radar, the scanning frequency is dynamically adjusted according to the moving speed of the detection module, when the moving speed v of the detection module is greater than a set threshold v0, the scanning frequency f is increased to n times of the original frequency, so as to ensure that the density of the collected point cloud data meets the accuracy requirement of the channel positioning and measurement based on the flange surface inclination angle a, and the set threshold v0 and the multiple n are preset.

[0182] According to a specific implementation manner of the embodiment of the present application, in step S3, the 2D area array camera lens adopts a fisheye lens, the distortion parameters of the fisheye lens are accurately obtained in the calibration process and are used for subsequent image processing, the collected channel image is subjected to de-distortion processing through a distortion correction algorithm, so that the channel edge profile is more accurate, and the calculation accuracy of the angle deflection amount and the Y-direction offset amount based on the accurate point position and size of the channel bolt is improved.

[0183] According to a specific implementation manner of the embodiment of the present application, in steps S1 and S2, the data collection of each sensor is implemented in hardware synchronization, each sensor collects data through the same clock source, so as to ensure that the consistency of the hoist column flange hole detection data and the channel point cloud data in time when the upper screw hole distance D1, the lower screw hole distance D2 and the flange surface inclination angle a and other data are obtained, and reduce the positioning error caused by different time.

[0184] According to a specific implementation manner of the embodiment of the present application, in step S2, when the point cloud data collected by the laser radar is subjected to de-noising processing, a bilateral filtering algorithm is adopted, which can remove noise while retaining the edge information of the point cloud, and the spatial distance weight function and the gray similarity weight function of the bilateral filtering are dynamically adjusted according to the point cloud density and the noise characteristics, so as to achieve the best de-noising effect and ensure the accuracy of the point cloud data for calculating the flange surface inclination angle a.

[0185] According to a specific implementation manner of the embodiment of the present application, in step S5, the mechanical arm walking process adopts a trajectory planning algorithm based on model predictive control (MPC), according to the calculated channel angle deflection and Y-direction offset, in combination with the mechanical arm dynamics model, the state of the mechanical arm at multiple future time points is predicted, and the control input of the mechanical arm is optimized at each sampling time, and in combination with the upper hole spacing D1, the lower hole spacing D2 and the flange surface inclination angle α, the mechanical arm is quickly and smoothly aligned with the channel and the column installation is completed, and the risk of collision is avoided.

[0186] According to a specific implementation manner of the embodiment of the present application, when a sensor fails, the system automatically switches to a redundant sensor working mode, ensuring the continuity of detection, positioning and guidance during the acquisition of key data such as the upper hole spacing D1, the lower hole spacing D2, the flange surface inclination angle α, the channel bolt accurate point and size.

[0187] According to a specific implementation manner of the embodiment of the present application, in step S1, the range of the laser sensor measuring distance is adjusted according to the size of the column flange and the installation distance of the detection module and the flange, so that the laser sensor can accurately measure the distance between the upper and lower detection positions, and the measurement accuracy is better than ±0.5mm, which meets the high-precision requirements of accurately calculating the upper hole spacing D1 and subsequently calculating the flange surface inclination angle α based on the difference between L1 and L2.

[0188] According to a specific implementation manner of the embodiment of the present application, in step S2, the tunnel feature map is pre-constructed according to different tunnel types and past engineering data, and the feature map contains information such as tunnel profile and channel distribution. In the point cloud matching process, the corresponding feature map is selected according to the current tunnel type for matching, which improves the efficiency and accuracy of channel positioning based on the flange surface inclination angle α and other data.

[0189] The above is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto, any changes or replacements within the technical range disclosed by the present application can be easily thought by those skilled in the art, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A multi-sensor fusion detection, positioning and guidance method for tunnel suspender installation, characterized in that: The following steps are involved: S1: The system controls the lifting module to move the detection module containing two camera assemblies and a laser sensor to the upper detection position. The camera assembly collects images of the left and right screw holes, determines the screw hole shape through threshold binarization, contour extraction, and template matching, and locates the pixel coordinates (X1, Y1) and (X2, Y2) of the screw hole center. The laser sensor measures the distance. When the difference between the two measured values ​​is less than the set value, the distance average L1 is calculated. The magnification N1 is calculated by looking up the L1 table or polynomial, and the upper screw hole spacing D1 is calculated. S2: Based on the upper screw hole spacing D1 obtained in S1, the detection module is controlled to move vertically downward a distance H to the lower detection position. The camera component collects images of the left and right screw holes and repeats the image processing process of S1 to locate the pixel coordinates (X3, Y3) and (X4, Y4) of the screw hole center. The laser sensor measures the distance and calculates the average value L2. The lower screw hole spacing D2 is calculated, and the flange surface inclination angle α is calculated based on the difference between L1 and L2. S3: Based on the flange surface inclination angle α obtained in S2, the laser radar scans along a Z-shaped path to collect 3D point clouds of the groove, reflective stickers, and tunnels. Outliers are removed through statistical filtering, ground points are determined through ray-based ground filtering, and data is compressed by voxel downsampling. The point cloud is then converted to the carrier coordinate system. S4: Based on the point cloud data converted in S3, the current frame point cloud is matched with the tunnel feature map, the tunnel centerline and the channel space linear model are fitted, the target area point cloud is extracted, and the transformation matrix is ​​solved based on SVD decomposition to determine the precise position and size of the channel bolts. S5: Based on the precise location and size of the channel bolts determined in S4, the origin of the robot arm's tool coordinate system is calibrated using the three-point method. After the 2D area array camera and the robot arm's coordinates are calibrated, the channel image is captured and binarized, contour filled, and the minimum circumscribed rectangle is calculated to obtain the channel's angular deflection and Y-direction offset. The robot arm moves according to the offset and aligns with the channel. Combined with the upper screw hole spacing D1, lower screw hole spacing D2, and flange surface inclination angle α obtained in S1 and S2, the column is automatically installed.

2. The multi-sensor fusion detection, positioning, and guidance method for tunnel suspender installation according to claim 1 is characterized in that: In step S1, when the camera component captures images, the exposure time is adaptively adjusted according to the light intensity in the tunnel. The light intensity is obtained through the ambient light sensor installed on the detection module. The exposure time t and the light intensity I satisfy the functional relationship t=k / I, where k is a preset constant to ensure that the captured screw hole image is clear and improve the accuracy of the screw hole center pixel coordinate positioning.

3. The multi-sensor fusion detection, positioning and guidance method for tunnel suspender installation according to claim 2 is characterized in that: In step S2, during the laser radar scanning process, the scanning frequency is dynamically adjusted according to the moving speed of the detection module. When the moving speed v of the detection module is greater than the set threshold v0, the scanning frequency f is increased to n times the original frequency to ensure that during the movement of the detection module, the density of the collected point cloud data meets the requirements for groove positioning and measurement accuracy based on the flange surface inclination angle α. The set threshold v0 and the multiple n are pre-set.

4. The multi-sensor fusion detection, positioning, and guidance method for tunnel suspender installation according to claim 3 is characterized in that: In step S3, the 2D area array camera lens adopts a fisheye lens. The distortion parameters of the fisheye lens are accurately obtained during the calibration process and used for subsequent image processing. The collected groove image is dedistorted by the distortion correction algorithm to make the groove edge contour more accurate, thereby improving the calculation accuracy of the angular deflection and Y-axis offset based on the precise position and size of the groove bolts.

5. The multi-sensor fusion detection, positioning and guidance method for tunnel suspender installation according to claim 4 is characterized in that: In steps S1 and S2, the data collection of each sensor is synchronized by hardware, and the data collection of each sensor is triggered by the same clock source to ensure that when obtaining data such as the upper screw hole spacing D1, the lower screw hole spacing D2, and the flange surface inclination angle α, the column flange screw hole detection data and the groove point cloud data are consistent in time, thereby reducing the positioning error caused by time asynchrony.

6. The multi-sensor fusion detection, positioning and guidance method for tunnel suspender installation according to claim 5, characterized in that: In step S2, the bilateral filtering algorithm is used to denoise the point cloud data collected by the lidar. This algorithm retains the edge information of the point cloud while removing noise. The spatial distance weight function and grayscale similarity weight function of the bilateral filter are dynamically adjusted according to the point cloud density and noise characteristics to achieve the best denoising effect and ensure the accuracy of the point cloud data for calculating the flange surface inclination angle α.

7. The multi-sensor fusion detection, positioning, and guidance method for tunnel suspender installation according to claim 6, characterized in that: In step S5, the robot arm movement process adopts a trajectory planning algorithm based on model predictive control (MPC). According to the calculated groove angle deflection and Y-axis offset, combined with the robot arm dynamics model, the robot arm's state at multiple moments in the future is predicted, and the robot arm's control input is optimized at each sampling moment. At the same time, combined with the upper screw hole spacing D1, the lower screw hole spacing D2 and the flange surface inclination angle α, the robot arm can quickly and smoothly align with the groove and complete the column installation to avoid collision risks.

8. The multi-sensor fusion detection, positioning, and guidance method for tunnel suspender installation according to claim 1, characterized in that: When a sensor fails, the system automatically switches to the redundant sensor working mode to ensure the continuity of detection, positioning and guidance work in the process of obtaining key data such as the upper screw hole spacing D1, the lower screw hole spacing D2, the flange surface inclination angle α, the precise position and size of the channel bolts.

9. The multi-sensor fusion detection, positioning, and guidance method for tunnel suspender installation according to claim 1, characterized in that: In step S1, the range of the laser sensor's distance measurement is adjusted according to the size of the column flange and the installation distance between the detection module and the flange, so that the laser sensor can accurately measure the distance between the upper and lower detection positions, and the measurement accuracy is better than ±0.5mm, meeting the high-precision requirements of accurately calculating the upper screw hole spacing D1 and the subsequent calculation of the flange surface inclination angle α based on the difference between L1 and L2.

10. The multi-sensor fusion detection, positioning and guidance method for tunnel suspender installation according to claim 1, characterized in that: In step S2, a tunnel feature map is pre-constructed based on different tunnel types and previous engineering data. The feature map contains information such as tunnel contour and groove distribution. During the point cloud matching process, the corresponding feature map is selected for matching based on the current tunnel type to improve the efficiency and accuracy of groove positioning based on data such as the flange surface inclination angle α.

Images