Positioning system for construction space in tunnel

Abstract

The invention provides a positioning system for construction space in a tunnel, and the system comprises an acquisition module which is used for obtaining the depth image information of a bolt group at the top of the tunnel; the positioning module is used for positioning the position of the bolt group based on the acquired depth image information; calculating and identifying the angle of the bolt group and the offset of the angle of the bolt group based on the position information of the bolt group; and the measuring module is used for obtaining the depth value of the root of the bolt based on the bolt area obtained by the positioning module, extracting the background of the root of the bolt based on the root value of the bolt and calculating the circular points of the root of all the bolts so as to obtain all to-be-measured sizes, thereby improving the accuracy of mounting the davit.

Description

Technical Field

[0001] The invention relates to the field of tunnel top suspension column positioning, and in particular to a construction space positioning system in a tunnel. Background Art

[0002] When installing the suspender in a high-speed railway tunnel, the bolts are fixed to the inner wall of the tunnel and then installed by the mechanical arm of the installation equipment. During the installation process, it is necessary to accurately determine the starting point, longitudinal measurement, and transverse measurement. If the measurement is inaccurate, it may cause the position of the positioning bolt to deviate, affecting the installation accuracy of the suspender.

[0003] When installing suspenders in railway tunnels, traditional positioning and measurement methods have some limitations. These methods usually rely on indirect methods such as hanging plumbs from the top of the tunnel or using laser contact network detectors, total stations, etc. for positioning. These methods are not only complicated and inefficient in the measurement process, but also require multiple people to work together, increasing construction costs and time. In addition, the top of railway tunnels is mostly arc-shaped, and the fixed base plate of the suspender needs to form a certain angle with the horizontal plane during installation to ensure that the suspender is perpendicular to the track surface after installation. The angles between the fixed base plates of suspenders at different positions and the horizontal plane are different. Therefore, in addition to determining the installation position of the suspender, it is also necessary to measure the angle between the fixed base plate and the horizontal plane. However, in the prior art, the measurement of studs has certain limitations and low accuracy. Summary of the invention

[0004] In order to overcome the deficiencies of the prior art, the present invention provides a construction space positioning system in a tunnel.

[0005] In order to achieve the above object, the present invention provides a tunnel construction space positioning system, which includes:

[0006] An acquisition module is used to obtain the depth image information of the bolt group at the top of the tunnel;

[0007] A positioning module locates the position of the bolt group based on the acquired depth image information; and calculates and identifies the angle of the bolt group and the offset of the angle of the bolt group based on the position information of the bolt group;

[0008] The measurement module obtains the bolt root depth value based on the bolt area obtained by the positioning module, extracts the bolt root background based on the bolt root value, and calculates the circular points of all bolt roots to obtain all dimensions to be measured.

[0009] Preferably, the step of the positioning module identifying the position of the bolt group includes:

[0010] S1: The robot moves to the area to be positioned;

[0011] S2: PLC sends positioning command to the positioning system;

[0012] S3: After receiving the positioning command, the positioning system triggers the acquisition module to acquire the current position image, identify the xy position of the center point of the bolt group in the image, and calculate the offset of the xy coordinates of the center point of the bolt group to the center of the image;

[0013] S4: Determine whether the x-coordinate offset and the y-coordinate offset are respectively less than the set thresholds. If both the x-coordinate offset and the y-coordinate offset are less than the corresponding set thresholds, execute step S5;

[0014] If at least one of the x-coordinate offset or the y-coordinate offset is greater than the set threshold, the xy-coordinate offset information is returned to the PLC, and the PLC controls the robot arm to translate according to the xy-coordinate offset, and returns to step S2;

[0015] S5: Detect the angle of the bolt group and calculate the offset of the angle of the bolt group;

[0016] S6: Determine whether the angle offset is less than the set threshold. If the angle offset is less than the set threshold, send a positioning completion signal to the PLC to end the positioning process.

[0017] If it is greater than the set threshold, the angle offset information is returned to the PLC, and the PLC controls the robot arm to rotate according to the angle offset and returns to step S2 for repositioning.

[0018] Preferably, in step S3, the specific steps of identifying the xy position of the center point of the bolt group and calculating the offset include:

[0019] S3.1: Scan the depth image at the current position and obtain the pixel value of each pixel;

[0020] S3.2: Filter the noise of the depth image and use dynamic thresholding to segment the foreground and background of the depth image;

[0021] S3.3: Calculate the length and width of all foreground areas, and filter out all bolt areas in the foreground according to the length and width thresholds. If the number of filtered areas is greater than 0, execute the next step. Otherwise, it means that no bolts are detected, and a signal that no bolts are detected is returned.

[0022] S3.4: Fit the minimum circumscribed rectangle of the entire bolt group based on the areas of all bolts;

[0023] S3.5: Determine whether the number of screened areas is equal to the set number of bolts. If so, proceed to step S3.6; otherwise, proceed to step S3.9;

[0024] S3.6: Segment the top area of ​​each bolt;

[0025] S3.7: Fit the center of the top of the bolt;

[0026] S3.8: Calculate the center point of the entire bolt group;

[0027] S3.8: Calculate the x and y offsets of the robot translation by the distance difference between the coordinates x and y of the center point of the rectangular area and the x and y of the center point of the image respectively.

[0028] Preferably, in step S3.2, the specific steps of segmenting the foreground and background of the depth image include:

[0029] For each pixel in the image, select an N*N neighborhood around the current pixel, where N>1 and is an odd number;

[0030] Extract all pixel points with non-zero pixel values ​​in the corresponding neighborhood, and calculate the extreme difference values ​​of all non-zero pixels;

[0031] Determine whether the extreme difference value is within the set threshold range. If so, extract the position of the current pixel;

[0032] Repeat the above process until all pixels in the image are processed and all the pixels extracted are the contour points where the foreground and background meet.

[0033] Fill all the pixels inside the closed contour according to the closed contour, and the foreground area of ​​the entire area is filled.

[0034] Preferably, in step S3.6, the step of segmenting the top area of ​​each bolt comprises:

[0035] According to the center point of the minimum circumscribed rectangle of the entire bolt group calculated in step S3.4, calculate the point a in each bolt area that is farthest from the circumscribed rectangle, and draw a straight line between the farthest point and the center point of the rectangle;

[0036] Extract point a:

[0037] Set the point in the bolt area to P i =(x i ,y i ), the center point of the minimum circumscribed rectangle of the bolt group is P o =(x o ,y o ), calculate the distance d between two points oi :

[0038]

[0039] Traverse all bolt areas, traverse all points in each bolt area, and find the distance d in each area oi The largest point is point a;

[0040] According to the diameter D of the bolt, take point b on the same straight line with a distance D from the farthest point, and intercept the bolt area. The intercepted area is the top area of ​​the bolt.

[0041] Preferably, in step S3.8, the specific steps of calculating the center point of the entire bolt group include:

[0042] Set the coordinate positions of the stud vertices to: C1(x1,y1), C2(x2,y2), ..., Cn(x n ,y n ), n = 6 or 4, find the smallest and largest X coordinates and the smallest and largest Y coordinates. When n = 6, the calculation formula is:

[0043] x min =min(x1,x2,x3,x4,x5,x6),

[0044] x max =max(x1,x2,x3,x4,x5,x6),

[0045] y min =min(y1,y2,y3,y4,y5,y6),

[0046] y max =max(y1,y2,y3,y4,y5,y6),

[0047] Calculate the center point coordinates Cm(x m ,y m ):

[0048]

[0049] Preferably, the specific steps of identifying the bolt group angle in step S5 include:

[0050] S5.1: According to the detection result of the positioning module, determine whether the number of bolts is 4. If it is 4, proceed to step S5.2; otherwise, proceed to step S5.3;

[0051] S5.2: Calculate the angle of the six bolts. The steps include:

[0052] According to the minimum bounding rectangle output by the positioning module, extract the length and width of the rectangle as (w, h), and calculate the average length l of w and h:

[0053]

[0054] by As elements, close the bolt area and control the three bolts in a row to connect together to obtain area 1 and area 2;

[0055] Traverse the center points C1 to C6 on the top of the bolt, determine whether the center points C1 to C6 intersect with area 1 and area 2 respectively, and divide the center points that intersect with area 1 into group 1, and divide the center points that intersect with area 2 into group 2;

[0056] Fit two straight lines from all the points in group 1 and group 2 respectively, and calculate the angles A1 and A2 between the two straight lines and the image coordinate system respectively;

[0057] The overall angle A of the 6 bolts is the average of A1 and A2:

[0058]

[0059] S5.3: Calculate the overall angle A' of the four bolts, and extract the rectangular angle A' based on the minimum circumscribed rectangle obtained in the positioning step.

[0060] Preferably, the specific steps of measuring the bolt size by the measuring module include:

[0061] I: Sort by bolt position based on the bolt area obtained by the positioning module;

[0062] II: According to the center point of the minimum circumscribed rectangle of the entire bolt group area obtained by the positioning module, the pixel value corresponding to the center point coordinate is read as the depth value Rz corresponding to the center point. With Rz as the threshold, all the pixel points in the rectangular area whose depth values ​​meet the threshold range of [Rz-d, Rz+d] are extracted as all the pixel points at the root of the bolt. d is set by the host computer;

[0063] III: Convert all pixel coordinates and depth values ​​of the bolt root area into three-dimensional coordinates, and fit the root surface according to all three-dimensional coordinate points;

[0064] IV: Convert the position of the center point of the bolt root: According to the xyz coordinate values ​​of the center points of the top of all bolts, calculate the xyz coordinates of the projection point vertically projected to the root surface to obtain the xyz coordinates of the projection point, which is the xyz coordinate value of the center point of the root;

[0065] V: Calculate all dimensions to be measured;

[0066] VI: Send the calculation results to PLC.

[0067] Preferably, in step III, the specific steps of converting coordinate values ​​and fitting the root surface include:

[0068] The steps to convert the two-dimensional coordinates of the depth image into three-dimensional coordinates are:

[0069] Draw the field of view of the bolt, α is the field of view angle of the camera, D is the distance from the camera to the detection surface, D is the depth value of a certain pixel in the depth image, and L is the length of the field of view that the camera can capture at the current detection distance;

[0070] Assume the pixel coordinates of a point on the image are (x, y). Since the camera's field of view is divided into horizontal and vertical fields of view, the magnification of the camera's horizontal field of view is:

[0071]

[0072] Among them, L w is the lateral field of view magnification, D xy is the depth value corresponding to the pixel coordinate of the current image, α w is the lateral field of view;

[0073] The magnification of the vertical field of view is:

[0074]

[0075] Among them, L h is the vertical field magnification, D xy is the depth value corresponding to the pixel coordinate of the current image, α h is the vertical field of view;

[0076] Assuming the pixel resolution of the image is w*h, the three-dimensional coordinates corresponding to a point on the image are:

[0077]

[0078] z t =D,

[0079] Among them, x t is the x coordinate value of a point on the image, y t is the y coordinate value of a point on the image, z t is the z coordinate value of a point on the image;

[0080] The specific steps of surface fitting are:

[0081] Assume a set of 3D data points:

[0082] (x i ,y i ,z i ),

[0083] Where i = 1, 2, ..., N;

[0084] Suppose there is a surface:

[0085] S(x,y)=z;

[0086] Calculate the minimum sum of the squares of the perpendicular distances D from all points to the surface to obtain the optimal surface, that is:

[0087]

[0088] Where p is the parameter vector of the surface model.

[0089] Preferably, the calculation steps for all dimensions to be measured are:

[0090] Calculate the top height difference between every two bolts: subtract the depth value of the center of the top circle of the bolt from each other;

[0091] Calculate the height difference between the roots of every two bolts: It is calculated by subtracting the center depth value of the bolt root;

[0092] Calculate the top spacing between every two bolts: It is calculated from the distance between the center points of the top of the bolt and the center points of the circle;

[0093] Calculate the root spacing between every two bolts: It is calculated from the distance between the xy coordinates of the center point of the bolt root circle and the center point of the circle;

[0094] Calculate the exposed height of each bolt: subtract the root center depth value from the top center depth value of the same bolt.

[0095] The tunnel construction space positioning system provided by the present invention has the following beneficial effects:

[0096] The acquisition module of the system collects depth image information of the bolts on the top of the tunnel through a three-dimensional camera, and uses the two-dimensional image information through the positioning module to identify the exact position of the positioning bolt group; based on the positioning position, the measurement module converts the two-dimensional image information into three-dimensional image information, uses the three-dimensional image information features to identify the key points of the bolts (top point, root point, etc.), and measures the required dimensional information (bolt spacing, exposed height difference, etc.), thereby realizing the equipment's automatic positioning and automatic measurement functions for the bolts on the top of the tunnel, providing positioning information for subsequent hanging installation, and improving the accuracy of the installation of the hanging column. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 The diagram shows the arrangement of bolts and the actual sling panel in the experiment of the present invention; the left diagram shows the example arrangement of bolt groups in the experiment, and the right diagram shows the sling panel;

[0098] Figure 2 A general flow chart of the positioning bolt assembly provided by the present invention;

[0099] Figure 3 Bolt group center point identification flow chart provided by the present invention

[0100] Figure 4 The distribution diagram of the bolts provided by the present invention fixed on the top of the actual tunnel; the top of the bolts is the desired foreground;

[0101] Figure 5 A schematic diagram of the minimum circumscribed rectangular area of ​​the entire bolt group provided by the present invention;

[0102] Figure 6 A distance diagram between each bolt area and the center point of the minimum circumscribed rectangle provided by the present invention;

[0103] Figure 7 A schematic diagram of a bolt top area cut out according to the present invention;

[0104] Figure 8 A schematic diagram of calculating the coordinates of the center point C7 of the minimum circumscribed rectangle that can contain all the six center points of the top circles of the bolts provided by the present invention;

[0105] Fig. 9 A flow chart of bolt group angle identification provided by the present invention;

[0106] Fig.10 When calculating the overall angle of the root bolt provided by the present invention, the arrangement diagram of the bolt group of the example is as follows:

[0107] Fig.11 A schematic diagram of the coordinate positions corresponding to the arrangement diagram of the bolt group provided as an example of the invention;

[0108] Fig.12 A schematic diagram of a region where three bolt regions are connected when the bolt region is closed by the calculation provided by the present invention;

[0109] Fig.13 The position coordinates of the three bolts in the area connecting the areas where the three bolts are located;

[0110] Fig.14 A bolt size data diagram when measuring a bolt group provided by the present invention;

[0111] Fig.15 This is a flow chart of bolt group measurement provided by the present invention.

[0112] Fig.16 A schematic diagram of the bolt area obtained by positioning provided by the present invention;

[0113] Fig.17 A schematic diagram of converting two-dimensional coordinates of a depth image into three-dimensional coordinates provided by the present invention. DETAILED DESCRIPTION

[0114] The following describes the embodiments of the present invention by specific examples, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0115] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention, and thus the drawings only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0116] All directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, horizontal, vertical...) are only used to explain the relative position relationship, movement status, etc. between the components in a certain specific posture. If the specific posture changes, the directional indication will also change accordingly.

[0117] Due to installation errors and other reasons, the parallel relationship referred to in the embodiments of the present invention may actually be an approximately parallel relationship, and the vertical relationship may actually be an approximately vertical relationship.

[0118] The present invention provides a spatial positioning system for construction in a tunnel, which comprises: an acquisition module, a positioning module and a measurement module. The acquisition module is used to obtain the depth image information of the bolt group at the top of the tunnel. The positioning module locates the position of the bolt group based on the acquired depth image information; and calculates and identifies the angle of the bolt group and the offset of the bolt group angle based on the position information of the bolt group. The measurement module obtains the depth value of the bolt root based on the bolt area obtained by the positioning module, extracts the bolt root background based on the bolt root value, and calculates the circular points of all bolt roots to obtain all dimensions to be measured.

[0119] Specifically, the acquisition module of the system collects the depth image information of the bolts on the top of the tunnel through a three-dimensional camera, and uses the two-dimensional image information through the positioning module to identify the position of the positioning bolt group; based on the positioning position, the three-dimensional image information features are used to identify the key points of the bolts (top point, root point, etc.), and the required dimensional information (bolt spacing, exposed height difference, etc.) is measured, so as to realize the automatic positioning and automatic measurement functions of the equipment for the bolts on the top of the tunnel, and provide positioning information for subsequent hanging installation.

[0120] When the system is working, the steps are as follows (taking high-speed rail tunnel installation as an example):

[0121] The installation of the top suspender of the high-speed railway tunnel is done by manually pre-installing a set of bolts. The equipment drives to the designated installation point, and the equipment uses a mechanical arm to lift the 3D camera to the top of the tunnel to scan the top structure of the tunnel and identify the positioning bolt group;

[0122] After locating the bolt, measure the various dimensional data of the bolt, and determine whether it meets the installation conditions based on the measured data. If it does, the column is installed. If it does not, the relevant dimensional data that does not meet the conditions is returned, and the staff will adjust the bolt.

[0123] In the present invention, the bolt group consists of 4 or 6 bolts arranged in a square shape (2 bolts in a row or 3 bolts in a row, arranged in 2 rows), such as Figure 1 shown.

[0124] like Figure 2 As shown in the figure, when executing the positioning step, the overall process is:

[0125] S1: The robot moves to the area to be positioned;

[0126] S2: PLC sends positioning command to the positioning system;

[0127] S3: After receiving the positioning command, the positioning system triggers the camera to collect the current position image, identifies the xy position of the center point of the bolt group in the image, and calculates the offset of the xy coordinates of the center point of the bolt group to the center of the image;

[0128] S4: determining whether the x-coordinate offset and the y-coordinate offset are respectively smaller than a set threshold value; if the x-coordinate offset and the y-coordinate offset are both smaller than the respective set threshold values, executing step S5;

[0129] If at least one of the x-coordinate offset or the y-coordinate offset does not meet the set threshold, the xy-coordinate offset information is returned to the PLC (Programmable Logic Controller), and the PLC controls the robot arm to translate according to the xy offset, and then returns to step S2;

[0130] S5: Detect the angle of the bolt group and calculate the offset of the angle of the bolt group;

[0131] S6: Determine whether the angle offset is less than the set threshold. If the angle offset is less than the set threshold, send a positioning completion signal to the PLC to end the positioning process.

[0132] Otherwise, the angle offset information is returned to the PLC, and the PLC controls the robot arm to rotate according to the angle offset and then returns to step S2.

[0133] In step S3, when the xy position of the center point of the bolt group is identified, after the positioning system receives the positioning command from the PLC, step S3.1 is executed: scan the depth image of the current position and obtain the pixel value of each pixel. That is, the 3D camera is started to scan the depth image of the current position. The pixel value of each pixel in the depth image represents the distance value from the camera to the point. If the depth value of the pixel is 0, it means that the current position may be blocked or beyond the measurement distance, and the depth value cannot be obtained. Figure 3 As shown, after obtaining the depth image, step S3.2 is performed: filtering the noise of the depth image and using a dynamic threshold to segment the foreground and background of the depth image:

[0134] ①Filter to filter out noise;

[0135] ② Use dynamic thresholding to segment the image foreground and background.

[0136] Applying dynamic thresholding to segment regions at different distances in the depth image involves:

[0137] a. For each pixel in the image, select an N*N neighborhood around the pixel (N>1 and is an odd number);

[0138] b. Since there may be areas in the depth image where depth information cannot be obtained, when there is a 0 value in the neighborhood, adding a pixel with a pixel value of 0 to the calculation will result in a calculation error. In this step, the depth difference between different object planes is calculated, and the depth difference is used to determine whether the two planes are the foreground area and background area required for detection. Therefore, it is necessary to first extract all the pixels with non-zero pixel values ​​in the neighborhood and calculate the extreme difference values ​​of all non-zero pixels, that is, subtract the minimum value from the maximum value in the N*N neighborhood;

[0139] c. Determine whether the extreme difference value is within the set threshold range. If so, extract the position of the current pixel;

[0140] d. Repeat the above process until all pixels in the image are processed, and all the extracted pixels are the contour points at the junction of the foreground and the background;

[0141] The purpose of extracting foreground is: Figure 4 As shown, the top of the bolt is the desired foreground, that is, the 6 circular areas on the top, and the cement wall is the desired background. The purpose of segmenting the foreground and background is to identify the circle on the top of the bolt, and further use the center of the circle on the top of the bolt to fit the center points of the 6 bolts.

[0142] e. Fill all the pixels inside the closed contour according to the closed contour. After filling, the entire area becomes the foreground area.

[0143] S3.3: Calculate the length and width of all foreground areas, and filter out all bolt areas in the foreground according to the length and width thresholds. If the number of filtered areas is greater than 0, execute the next step. Otherwise, it means that no bolts are detected, and a signal that no bolts are detected is returned.

[0144] S3.4: Fit the minimum circumscribed rectangle of the entire bolt group based on the areas of all bolts;

[0145] The algorithm for multi-region fitting minimum enclosing rectangle is as follows:

[0146] a. Calculate the center of mass:

[0147] Centroid is the average position of all points and can be calculated using the following formula:

[0148]

[0149] Among them, (x i ,y i ) are the coordinates of the ith point, and n is the total number of points.

[0150] b. Construct the covariance matrix:

[0151] The covariance matrix is ​​used to find the main direction of the point set, and its formula is:

[0152]

[0153] Among them, σ xx is the variance in the x direction, σ yy is the variance in the y direction, σ xy and σ yx is the direct covariance of x and y, is the mean value in the corresponding direction.

[0154] c. Calculate the eigenvalues ​​and eigenvectors of the covariance matrix:

[0155] The eigenvalues ​​and eigenvectors can be obtained by solving the following characteristic equation:

[0156]

[0157] This will give two eigenvalues ​​λ1 and λ2, and the corresponding eigenvectors v1 and v2. The eigenvectors corresponding to the larger eigenvalues ​​represent the principal directions of the point set.

[0158] d. Determine the width and height of the minimum enclosing rectangle:

[0159] Use the eigenvalues ​​to determine the width and height of the minimum bounding rectangle:

[0160]

[0161] Where w is the width of the rectangle and l is the length of the rectangle.

[0162] e. Calculate the rotation angle:

[0163] The rotation angle θ can be calculated from the slope of the eigenvector:

[0164]

[0165] S3.5: Since a group of bolts consists of 4 or 6 bolts arranged in a square shape (2 bolts in a row or 3 bolts in a row, arranged in 2 rows), in order to prevent abnormal detection, the number of bolts is judged here to determine whether the number of screened areas is equal to 4 or 6. If so, proceed to step S3.6, otherwise proceed to step S3.9.

[0166] S3.6: Segment the top area of ​​each bolt;

[0167] Method for splitting the top of the bolt (taking 6 bolts as an example, the same applies to 4 bolts):

[0168] a. Based on the center point of the minimum circumscribed rectangle of the entire bolt group calculated in step S3.4, calculate the point a in each bolt area that is farthest from the circumscribed rectangle, and draw a straight line between the farthest point and the center point of the rectangle; Figure 5 shown.

[0169] The steps for extracting point a are:

[0170] Assume that the point in the bolt area is P i =(x i ,y i ), the center point of the minimum circumscribed rectangle of the bolt group is P o =(x o ,y o ), calculate the distance d between two points oi (Euclidean distance formula):

[0171]

[0172] Traverse all bolt areas, traverse all points in each bolt area, and find the points in each area such that d oi The largest point is point a. The schematic diagram is as follows Figure 6 shown.

[0173] b. According to the bolt diameter D, take the point b on the same straight line with a distance D from the farthest point to intercept the bolt area. The intercepted area is the bolt top area; Figure 7 shown.

[0174] S3.7: Fit the center of the top of the bolt;

[0175] Circle center fitting algorithm:

[0176] a. Set the circle equation to:

[0177] (xx c ) 2 +(yy c ) 2 =R 2 ,

[0178] Among them, (x c ,y c ) are the coordinates of the center of the circle, and R is the radius of the circle.

[0179] b. Set the objective function:

[0180]

[0181] Among them, f is the objective function which minimizes the sum of the squares of the algebraic distances of all data points to the circle. i ,y i ) is the coordinate of the i-th pixel.

[0182] c. Linearized equation:

[0183] Linearize the above equation and assume:

[0184] g(x,y)=(x c ) 2 +(yy c ) 2 -R 2 ,

[0185] Then the objective function can be expressed as:

[0186]

[0187] By minimizing f, we can find the optimal x c ,y c and R.

[0188] S3.8: Calculate the center point of the entire bolt group;

[0189] Center point calculation method (taking 6 bolts as an example, the same applies to 4 bolts):

[0190] like Figure 8 As shown in the figure, the coordinates (x) of the center point Cm of the smallest circumscribed rectangle that can contain all the points are calculated based on the center points C1 to C6 of the top of the six bolts (C1 to C4 for four bolts). m ,y m ), where, according to the arrangement order, when the number of bolts is 6, m is 7, and when the number of bolts is 4, m is 5.

[0191] The calculation method is as follows:

[0192] Set the coordinate positions of the stud vertices to: C1(x1,y1), C2(x2,y2), ..., Cn(x n ,y n ), n = 6 or 4; take 6 point coordinates as an example: Assume that the 6 point coordinates are C1(x1, y1), C2(x2, y2), C3(x3, y3), C4(x4, y4), C5(x5, y5), C6(x6, y6), respectively, find the smallest and largest X coordinates and the smallest and largest Y coordinates:

[0193] x min =min(x1,x2,x3,x4,x5,x6),

[0194] x max =max(x1,x2,x3,x4,x5,x6),

[0195] y min =min(y1,y2,y3,y4,y5,y6),

[0196] y max =max(y1,y2,y3,y4,y5,y6),

[0197] Calculate the center point coordinates C7(x7,y7):

[0198]

[0199] S3.9: Calculate the x and y offsets of the robot translation by the distance difference between the coordinates x and y of the center point of the rectangular area and the x and y of the center point of the image respectively.

[0200] like Fig. 9 As shown, the specific steps of identifying the bolt group angle in step S5 include:

[0201] S5.1: According to the detection result of the positioning module, determine whether the number of bolts is 4. If it is 4, proceed to step S5.3; otherwise, proceed to step S5.2;

[0202] S5.2: Calculate the angle of the six bolts as a whole. The calculation steps include:

[0203] like Fig.10 As shown in the figure, when the bolt group consists of 6 bolts, the chassis of the suspension column has 6 waist holes whose arrangement is consistent with the arrangement of the bolts. To ensure that the waist holes of the chassis can be aligned with the bolts, the calculation of the bolt group angle needs to be differentiated according to the arrangement of the bolts. The specific process is as follows:

[0204] a. According to the minimum enclosing rectangle output by the positioning module, the length and width of the extracted rectangle are (w, h) respectively. Theoretically, the minimum enclosing rectangle should be a square, but the actual fitting results w and h will have deviations in length, so first calculate the average length l of w and h:

[0205]

[0206] b. As elements, close the bolt area to connect the three bolts in a row to obtain area 1 and area 2; Fig.11 shown.

[0207] c. Traverse the center points C1 to C6 on the top of the bolt, and determine whether C1 to C6 intersect with area 1 and area 2 respectively. The center points that intersect with area 1 are grouped into group 1, and the center points that intersect with area 2 are grouped into group 2;

[0208] d. Fit two straight lines to all points in group 1 and group 2 respectively, and calculate the angles A1 and A2 between the two straight lines and the image coordinate system respectively; Fig.12 shown.

[0209] e. The overall angle A of the 6 bolts is the average of A1 and A2:

[0210]

[0211] S5.3: Calculate the overall angle A' of the four bolts, and extract the rectangular angle A' based on the minimum circumscribed rectangle obtained in the positioning step S7.

[0212] Based on the position information of the positioning bolts, the bolt group is moved as a whole to the center of the field of view of the image. After the positioning is completed, the PLC will send a measurement signal to the positioning system. The positioning system calculates various values ​​based on the current image and returns the values ​​to the PLC after the calculation is completed. The PLC will determine whether the size is qualified.

[0213] When measuring bolt groups, the bolt size data diagram is as follows: Fig.14 As shown. The green area represents the cross section of the suspender column, the light green vertical line represents the bolt, A is the root spacing, B is the top spacing, C is the root height difference, D is the top height difference, and E and F are the exposed lengths of the bolts.

[0214] like Fig.15 The steps to measure bolt size include:

[0215] I: Based on the bolt area obtained in the positioning step, sort the bolts from left to right and from top to bottom according to their positions. Fig.16 shown.

[0216] II: According to the center point of the minimum circumscribed rectangle of the entire area of ​​the bolt group obtained in the positioning step, read the pixel value corresponding to the coordinates of the center point, which is the depth value Rz corresponding to the center point. Take Rz as the threshold, extract all the pixel points in the rectangular area whose depth values ​​meet the threshold range of [Rz-d, Rz+d], which are all the pixel points at the root of the bolt. d is set by the host computer;

[0217] II: Convert all pixel coordinates and depth values ​​of the bolt root area into three-dimensional coordinates, and fit the root surface according to all three-dimensional coordinate points;

[0218] a. Method of converting 2D coordinates of depth image into 3D coordinates:

[0219] like Fig.17 As shown in the figure, α is the camera field of view angle, D is the distance from the 3D camera to the detection surface, D is the pixel value of a certain pixel in the depth image, that is, the depth value, and L is the length of the field of view that the camera can capture at the current detection distance.

[0220] Assume that the pixel coordinates of a point on the image are (x, y). Since the camera's field of view is divided into horizontal and vertical fields of view, the magnification of the camera's horizontal field of view is:

[0221]

[0222] Among them, L w is the lateral field of view magnification, D xy is the depth value corresponding to the pixel coordinate of the current image, α w is the lateral field of view.

[0223] The magnification of the vertical field of view is:

[0224]

[0225] Among them, L n is the vertical field magnification, D xy is the depth value corresponding to the pixel coordinate of the current image, α h is the vertical field of view.

[0226] Assuming that the pixel resolution of the image is w*h, the three-dimensional coordinates corresponding to a point on the image are:

[0227]

[0228] z t =D,

[0229] Among them, x t is the x coordinate value of a point on the image, y t is the y coordinate value of a point on the image, z t is the z-coordinate value of a point on the image.

[0230] b. Surface fitting method:

[0231] Suppose there is a set of three-dimensional data points:

[0232] (x i ,y i ,z i ),

[0233] Where i=1,2,...,N.

[0234] Assume there is a surface:

[0235] S(x,y)=z,

[0236] So that this surface can best fit these points.

[0237] Calculate the minimum sum of the squares of the vertical distances D from all points to the surface to obtain the optimal surface, that is:

[0238]

[0239] Where p is the parameter vector of the surface model.

[0240] IV: Convert the center point of the bolt root;

[0241] Conversion method of bolt root center point:

[0242] According to the xyz coordinate values ​​of the center points of the tops of all bolts, the xyz coordinates are vertically projected onto the projection point of the root surface to obtain the xyz coordinates of the projection point, which is the xyz coordinate value of the center point of the root.

[0243] V: Calculate all dimensions to be measured;

[0244] a. Calculate the top height difference between every two bolts: subtract the depth value of the center of the top circle of the bolt from each other;

[0245] b. Calculate the height difference between the roots of every two bolts: It is calculated by subtracting the center depth value of the bolt root;

[0246] c. Calculate the top spacing between every two bolts: It is calculated from the distance between the center points of the top of the bolt and the center points of the bolt;

[0247] d. Calculate the root spacing between every two bolts: calculated from the distance between the xy coordinates of the center point of the bolt root and the center point of the bolt root;

[0248] e. Calculate the exposed height of each bolt: the height is calculated by subtracting the center depth of the root from the center depth of the top of the bolt.

[0249] VI: Send the calculation results to PLC.

[0250] Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

Claims

1. The tunnel construction space positioning system is characterized by: include: An acquisition module is used to obtain the depth image information of the bolt group at the top of the tunnel; A positioning module locates the position of the bolt group based on the acquired depth image information; Calculate and identify the angle of the bolt group and the offset of the angle of the bolt group based on the position information of the bolt group; The measurement module obtains the bolt root depth value based on the bolt area obtained by the positioning module, extracts the bolt root background based on the bolt root value, and calculates the circular points of all bolt roots to obtain all dimensions to be measured.

2. The tunnel construction space positioning system according to claim 1, characterized in that: The step of the positioning module identifying the position of the bolt group includes: S1: The robot moves to the area to be positioned; S2: PLC sends positioning command to the positioning system; S3: After receiving the positioning command, the positioning system triggers the acquisition module to acquire the current position image, identify the xy position of the center point of the bolt group in the image, and calculate the offset of the xy coordinates of the center point of the bolt group to the center of the image; S4: Determine whether the x-coordinate offset and the y-coordinate offset are respectively less than the set thresholds. If both the x-coordinate offset and the y-coordinate offset are less than the corresponding set thresholds, execute step S5; If at least one of the x-coordinate offset or the y-coordinate offset is greater than the set threshold, the xy-coordinate offset information is returned to the PLC, and the PLC controls the robot arm to translate according to the xy-coordinate offset, and returns to step S2; S5: Detect the angle of the bolt group and calculate the offset of the angle of the bolt group; S6: Determine whether the angle offset is less than the set threshold. If the angle offset is less than the set threshold, send a positioning completion signal to the PLC to end the positioning process. If it is greater than the set threshold, the angle offset information is returned to the PLC, and the PLC controls the robot arm to rotate according to the angle offset and returns to step S2 for repositioning.

3. The tunnel construction space positioning system according to claim 2, characterized in that: In step S3, the specific steps of identifying the xy position of the center point of the bolt group and calculating the offset include: S3.1: Scan the depth image at the current position and obtain the pixel value of each pixel; S3.2: Filter the noise of the depth image and use dynamic thresholding to segment the foreground and background of the depth image; S3.3: Calculate the length and width of all foreground areas, and filter out all bolt areas in the foreground according to the length and width thresholds. If the number of filtered areas is greater than 0, execute the next step. Otherwise, it means that no bolts are detected, and a signal that no bolts are detected is returned. S3.4: Fit the minimum circumscribed rectangle of the entire bolt group based on the areas of all bolts; S3.5: Determine whether the number of screened areas is equal to the set number of bolts. If so, proceed to step S3.6; otherwise, proceed to step S3.9; S3.6: Segment the top area of ​​each bolt; S3.7: Fit the center of the top of the bolt; S3.8: Calculate the center point of the entire bolt group; S3.9: Calculate the x and y offsets of the robot translation by the distance difference between the coordinates x and y of the center point of the rectangular area and the x and y of the center point of the image respectively.

4. The tunnel construction space positioning system according to claim 3, characterized in that: In step S3.2, the specific steps of segmenting the depth image foreground and background include: For each pixel in the image, select an N*N neighborhood around the current pixel, where N>1 and is an odd number; Extract all pixel points with non-zero pixel values ​​in the corresponding neighborhood, and calculate the extreme difference values ​​of all non-zero pixels; Determine whether the extreme difference value is within the set threshold range. If so, extract the position of the current pixel; Repeat the above process until all pixels in the image are processed and all the pixels extracted are the contour points where the foreground and background meet. Fill all the pixels inside the closed contour according to the closed contour, and the foreground area of ​​the entire area is filled.

5. The tunnel construction space positioning system according to claim 3, characterized in that: In step S3.6, the step of segmenting the top area of ​​each bolt includes: According to the center point of the minimum circumscribed rectangle of the entire bolt group calculated in step S3.4, calculate the point a in each bolt area that is farthest from the circumscribed rectangle, and draw a straight line between the farthest point and the center point of the rectangle; Extract point a: Set the point in the bolt area to P i =(x i ,y i ), the center point of the minimum circumscribed rectangle of the bolt group is P o =(x o ,y o ), calculate the distance d between two points oi : Traverse all bolt areas, traverse all points in each bolt area, and find the distance d in each area oi The largest point is point a; According to the diameter D of the bolt, take point b on the same straight line with a distance D from the farthest point, and intercept the bolt area. The intercepted area is the top area of ​​the bolt.

6. The tunnel construction space positioning system according to claim 3, characterized in that: In step S3.8, the specific steps of calculating the center point of the entire bolt group include: Set the coordinate positions of the stud vertices to: C1(x1,y1), C2(x2,y2), ..., Cn(x n ,y n ), n = 6 or 4, find the smallest and largest X coordinates and the smallest and largest Y coordinates. When n = 6, the calculation formula is: x min =min(x1,x2,x3,x4,x5,x6), x max =max(x1,x2,x3,x4,x5,x6), <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> min <h2 style=";text-align:left;direction:ltr"> = min(y1,y2,y3,y4,y5,y6) <h2 style=";text-align:left;direction:ltr">y<h2 style=";text-align:left;direction:ltr"> max <h2 style=";text-align:left;direction:ltr"> = max(y1,y2,y3,y4,y5,y6) Calculate the center point coordinates Cm(x m ,y m ):

7. The tunnel construction space positioning system according to claim 1, characterized in that: The specific steps of identifying the bolt group angle in step S5 include: S5.1: According to the detection result of the positioning module, determine whether the number of bolts is 4. If it is 4, proceed to step S5.3; otherwise, proceed to step S5.2; S5.2: Calculate the angle of the six bolts. The steps include: According to the minimum bounding rectangle output by the positioning module, extract the length and width of the rectangle as (w, h), and calculate the average length l of w and h: by As elements, close the bolt area and control the three bolts in a row to connect together to obtain area 1 and area 2; Traverse the center points C1 to C6 on the top of the bolt, determine whether the center points C1 to C6 intersect with area 1 and area 2 respectively, and divide the center points that intersect with area 1 into group 1, and divide the center points that intersect with area 2 into group 2; Fit two straight lines from all the points in group 1 and group 2 respectively, and calculate the angles A1 and A2 between the two straight lines and the image coordinate system respectively; The overall angle A of the 6 bolts is the average of A1 and A2: S5.3: Calculate the overall angle A' of the four bolts, and extract the rectangular angle A' based on the minimum circumscribed rectangle obtained in the positioning step.

8. The tunnel construction space positioning system according to claim 1, characterized in that: The specific steps of measuring the bolt size by the measuring module include: I: Sort by bolt position based on the bolt area obtained by the positioning module; II: According to the center point of the minimum circumscribed rectangle of the entire bolt group area obtained by the positioning module, the pixel value corresponding to the center point coordinate is read as the depth value Rz corresponding to the center point. With Rz as the threshold, all the pixel points in the rectangular area whose depth values ​​meet the threshold range of [Rz-d, Rz+d] are extracted as all the pixel points at the root of the bolt. d is set by the host computer; II: Convert all pixel coordinates and depth values ​​of the bolt root area into three-dimensional coordinates, and fit the root surface according to all three-dimensional coordinate points; IV: Convert the position of the center point of the bolt root: According to the xyz coordinate values ​​of the center points of the top of all bolts, calculate the xyz coordinates of the projection point vertically projected to the root surface to obtain the xyz coordinates of the projection point, which is the xyz coordinate value of the center point of the root; V: Calculate all dimensions to be measured; VI: Send the calculation results to PLC.

9. The tunnel construction space positioning system according to claim 8, characterized in that: In step III, the specific steps of converting coordinate values ​​and fitting the root surface include: The steps to convert the two-dimensional coordinates of the depth image into three-dimensional coordinates are: Draw the field of view of the bolt, α is the field of view angle of the camera, D is the distance from the camera to the detection surface, D is the depth value of a certain pixel point in the depth image, and L is the length of the field of view that the camera can capture at the current detection distance; Assume the pixel coordinates of a point on the image are (x, y). Since the camera's field of view is divided into horizontal and vertical fields of view, the magnification of the camera's horizontal field of view is: Among them, L w is the lateral field of view magnification, D xy is the depth value corresponding to the pixel coordinate of the current image, α w is the lateral field of view; The magnification of the vertical field of view is: Among them, L h is the vertical field magnification, D xy is the depth value corresponding to the pixel coordinate of the current image, α h is the vertical field of view; Assuming the pixel resolution of the image is w*h, the three-dimensional coordinates corresponding to a point on the image are: z t =D, Among them, x t is the x coordinate value of a point on the image, y t is the y coordinate value of a point on the image, z t is the z coordinate value of a point on the image; The specific steps of surface fitting are: Assume a set of 3D data points: (x i ,y i ,z i ), Where i = 1, 2, ..., N; Suppose there is a surface: S(x,y)=z; Calculate the minimum sum of the squares of the perpendicular distances D from all points to the surface to obtain the optimal surface, that is: Where p is the parameter vector of the surface model.

10. The tunnel construction space positioning system according to claim 8, characterized in that: The calculation steps for all the dimensions to be measured are: Calculate the top height difference between every two bolts: subtract the depth value of the center of the top circle of the bolt from each other; Calculate the height difference between the roots of every two bolts: It is calculated by subtracting the center depth value of the bolt root; Calculate the top spacing between every two bolts: It is calculated from the distance between the center points of the top of the bolt and the center points of the circle; Calculate the root spacing between every two bolts: It is calculated from the distance between the xy coordinates of the center point of the bolt root circle and the center point of the circle; Calculate the exposed height of each bolt: subtract the root center depth value from the top center depth value of the same bolt.

Images