Multi-view speckle laser image measurement method for grabbing and positioning tunnel segment

Abstract

The invention provides a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments. The method comprises the following steps: arranging a speckle array laser and a monocular camera on a grabbing disc at the tail end of a segment erector; when a translation mechanism of a segment erector moves towards a segment to be grabbed in the shield tail direction, a speckle array laser is used for emitting speckle laser, and a monocular camera is used for continuously collecting speckle images of the segment to be grabbed and a hoisting bolt; an improved YOLOV9 model is adopted to carry out contour detection and recognition on the to-be-grabbed segment and the lifting bolt in the speckle image, and the contours of the to-be-grabbed segment and the lifting bolt are extracted; and the contour of the hoisting bolt is processed by adopting an ellipse detection algorithm of arc clustering to determine the coordinates of the central point of the hoisting bolt on the segment to be grabbed, so that the relative position variation of the central point of the hoisting bolt in the speckle images of adjacent frames is obtained. High-precision positioning and path guiding are provided for a tunnel segment assembling mechanism, and corresponding theoretical and technical supports are provided for automatic assembling of tunnel linings.

Description

Technical Field

[0001] The invention relates to the technical field of tunnel lining construction segment assembly positioning, and in particular to a multi-viewing angle speckle laser image measurement method for tunnel segment grabbing and positioning. Background Art

[0002] In recent years, with the introduction of the concept of Industry 4.0, society has entered an era where information technology promotes industrial transformation, that is, the era of intelligence. As a major national weapon, the shield machine is currently widely used in the construction of tunnels, subways, urban roads and bridges, and is an important part of infrastructure construction. Therefore, it is critical to improve the working efficiency and quality of the shield machine. As an important part of the shield machine's work, the segment assembly machine can effectively improve the accuracy of grabbing the segments, shorten work efficiency, and reduce labor costs and risk accidents. It is of great significance for automated assembly.

[0003] In theory, segment assembly is to assemble the tunnel segment rings of the tunneled section by the segment assembler while the shield head of the shield machine is excavating, so as to ensure the safety and stability of the tunnel. The working efficiency of the segment assembler can be effectively improved by accurately identifying the lifting bolts on the segments to be grabbed.

[0004] The invention with application number 202211490907.1 discloses an automatic assembly system for capping segments based on deep learning and laser. The trained model YOLOv5s is selected, and the groove target of the capping segment is identified and located by a monocular vision intelligent camera, and then the relative position of the assembler and the capping segment is calculated, the movement of the assembler is controlled, and the automatic grabbing of the capping segment by the assembler is completed with the help of laser ranging; then the groove target of the assembled segment is identified by a monocular vision intelligent camera, and the relative position of the capping segment and the assembled segment is calculated, and the automatic assembly of the assembler is controlled. Finally, the automatic grabbing and assembly of the capping segment is realized. The above invention solves the problem of high precision requirements for the capping segments, so that the entire process of automatic assembly of the capping segments is accurately positioned, time-saving, safe and stable. However, the above invention does not take into account the errors caused by processing images using digital image technology. Summary of the invention

[0005] In order to overcome the defects of inaccurate positioning of lifting bolts and assembly error detection in the existing shield construction process and improve work efficiency, the present invention proposes a multi-view speckle laser image measurement method for tunnel segment grabbing and positioning.

[0006] In order to achieve the above object, the technical solution of the present invention is achieved as follows:

[0007] A multi-view speckle laser image measurement method for tunnel segment grabbing and positioning, comprising the following steps:

[0008] S1. Arrange and install a visual analysis device on the end grabbing plate of the segment assembly machine of the shield machine, the visual analysis device including a speckle array laser and a monocular camera;

[0009] S2. When the translation mechanism of the segment assembly machine moves toward the segment to be grabbed in the direction of the shield tail, the speckle array laser is used to emit speckle laser, and the monocular camera is used to continuously collect speckle images of the segment to be grabbed and the lifting bolts;

[0010] S3. Use the improved YOLOV9 model to detect and identify the contours of the segments to be grabbed and the lifting bolts in the speckle image, and extract the contours of the segments to be grabbed and the lifting bolts;

[0011] S4. The contour of the lifting bolt is processed by the ellipse detection algorithm of arc clustering to determine the coordinates of the center point of the lifting bolt on the pipe segment to be grasped. The coordinate transformation information of the image coordinates of the center point of the lifting bolt and the world coordinate system is obtained by subtracting the coordinate information of adjacent frames after coordinate conversion, thereby obtaining the relative position change of the center point of the lifting bolt in the speckle image of adjacent frames, which provides precise positioning and path guidance for the assembly mechanism to grasp the pipe segment.

[0012] There are four groups of speckle array lasers, including a first speckle array laser, a second speckle array laser, a third speckle array laser, and a fourth speckle array laser; the first speckle array laser is fixed at the middle of the left edge of a grabbing disk at the end of a segment assembly machine, the second speckle array laser is fixed at the middle of the left front end of a grabbing disk at the end of a segment assembly machine, the third speckle array laser is fixed at the middle of the right front end of a grabbing disk at the end of a segment assembly machine, and the fourth speckle array laser is fixed at the middle of the right edge of a grabbing disk at the end of a segment assembly machine.

[0013] There are six monocular cameras, including a first monocular camera, a second monocular camera, a third monocular camera, a fourth monocular camera, a fifth monocular camera, and a sixth monocular camera; the first monocular camera is fixed at the upper left corner of the left edge of the grabbing disk at the end of the segment assembly machine, the second monocular camera is fixed at the lower right corner of the left edge of the grabbing disk at the end of the segment assembly machine, the third monocular camera is fixed at the right side of the left front end of the grabbing disk at the end of the segment assembly machine, the fourth monocular camera is fixed at the left side of the right front end of the grabbing disk at the end of the segment assembly machine, the fifth monocular camera is fixed at the upper right corner of the right edge of the grabbing disk at the end of the segment assembly machine, and the sixth monocular camera is fixed at the lower right corner of the right edge of the grabbing disk at the end of the segment assembly machine.

[0014] When the monocular cameras are used to collect speckle images of the pipe segments to be grasped and the lifting bolts, the six monocular cameras are respectively combined into five groups of binocular cameras for image collection: the first monocular camera and the second monocular camera; the second monocular camera and the third monocular camera; the third monocular camera and the fourth monocular camera; the fourth monocular camera and the fifth monocular camera; the fifth monocular camera and the sixth monocular camera, and the speckle images collected by adjacent monocular camera groups overlap with each other.

[0015] The first monocular camera collects speckle images of the 0-1m area on the left edge of the pipe segment to be grasped, the second monocular camera collects speckle images of the 0.8-1.2m area on the left edge and the 0-0.8m area on the front edge of the pipe segment to be grasped, the third monocular camera collects speckle images of the 0.6-1.8m area on the front edge of the pipe segment to be grasped, the fourth monocular camera collects speckle images of the 1.6-2.8m area on the front edge of the pipe segment to be grasped, the fifth monocular camera collects speckle images of the 2.6-3.4m area on the front edge and the 0.8-1.2m area on the right edge of the pipe segment to be grasped, and the sixth monocular camera collects speckle images of the 0-1m area on the right edge of the pipe segment to be grasped.

[0016] The improved YOLOV9 model is obtained by introducing a self-attention mechanism into the backbone network layer of the YOLOV9 model; adding a terminal output layer network LOG output layer combined with LOG spot detection into the YOLOV9 model; and replacing the loss function CIoU Loss function in the original network structure of the YOLOV9 model with the EIoU Loss function.

[0017] The method of outputting the outline of the pipe segment and the lifting bolt to be grabbed by the LOG output layer is: Gaussian blur is applied to the feature image input to the LOG output layer by the LoG operator, and then Laplace calculation is performed. The LoG operator is: When the scale σ of LoG matches the width of the spot (the spatial size of the spot in the image), the amplitude of LoG is maximized at the center of the spot. Then, LoG operators of different scales are used to convolve with the signal to find the maximum amplitude of LoG. The point that reaches the maximum or minimum value in space and scale is the desired spot. The adjacent desired spots are connected in sequence to obtain the contour features existing in the entire speckle image, and the contours of the pipe segments and lifting bolts to be grabbed are extracted. Among them, G σ (x, y) is the value of the Gaussian function at position (x, y), x and y represent the coordinates of the edge pixel points, σ ​​is the standard deviation and scale of the Gaussian function, exp is the power operation of the base e of the natural logarithm, Denotes Laplace transform, G σ (x,y) represents the value of the Gaussian function at position (x,y), and I(x,y) represents the coordinates of the edge pixel.

[0018] The method for determining the center point coordinates of the lifting bolt on the pipe segment to be grasped is: arc clustering of the ellipses on the lifting bolt contour to determine the complete ellipse; extracting a directed boundary sequence for the identified complete ellipse, and fitting the ellipse parameters using the least squares method; screening the identified lifting bolt ellipses to obtain the required lifting bolt top surface ellipse, and the center point coordinates of the lifting bolt top surface ellipse are the center point coordinates of the lifting bolt.

[0019] The method for clustering arcs of ellipses on the contour of lifting bolts is as follows: taking the double endpoints of the arcs to be clustered in the contour map of the pipe segment to be grasped and the lifting bolt output by the improved YOLOV9 model as the starting point, emitting two rays in the same direction as the tangent direction of the arc to form an effective search range, and the arcs searched within the effective search range are the same ellipse; and taking the average position of the two points on the arc as the chord center, judging the distance between the arc center and the chord center, the arc center and the chord center should be the maximum value, and at this time, the arc retained in the effective search range is selected, otherwise it is discarded; judging the distance of the center of the fitted ellipse is judged based on the characteristics of the ellipse polar line, determining the tangent of the arc segment, and then defining the distance between the arc segment and the center of the ellipse, and selecting the diameter of the largest arc in the lifting bolt as the judgment threshold for comparison, if the distance between the arc segment and the center of the ellipse is less than the judgment threshold, it is determined to be part of the ellipse, otherwise it is discarded.

[0020] An image storage module and an image processing module are also arranged and installed on the end grabbing disk of the segment assembly machine of the shield machine. The image storage module is fixed on the left side of the third monocular camera, and the image processing module is fixed on the right side of the fourth monocular camera; the first monocular camera, the second monocular camera, the third monocular camera, the fourth monocular camera, the fifth monocular camera, and the sixth monocular camera are all connected to the image storage module, and the image storage module is connected to the image processing module.

[0021] Compared with the prior art, the present invention has the following beneficial effects:

[0022] 1. The hardware distribution provided by the present invention is arranged according to the "integration" principle, with fast data transmission speed, ingenious layout, and small hardware space occupation. Under the premise of not affecting the normal operation of the segment assembly machine, the working efficiency of the hardware is maximized.

[0023] 2. When arranging four groups of speckle array lasers, the present invention fully considers the characteristics of speckle laser imaging. There is an intersection of speckle images between adjacent laser groups, which reduces the error caused by uneven laser irradiation.

[0024] 3. The six monocular cameras arranged in the present invention are respectively combined into five groups of binocular cameras for image acquisition, and the speckle images acquired by adjacent camera groups overlap with each other, thereby reducing measurement errors.

[0025] 4. The present invention utilizes the characteristics of speckle images, collects the generated point cloud images, and then performs analysis and processing. The process is streamlined and efficient, and the measurement error is greatly reduced.

[0026] 5. The improved YOLOV9 detection model provided by the present invention solves the problem of balancing difficult and easy samples that is not considered in the original loss function.

[0027] 6. The improved YOLOV9 detection model provided by the present invention avoids the phenomenon of extremely poor training effect caused by not considering the relationship of the input layer. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 The present invention discloses an arrangement diagram of four groups of speckle array lasers, six monocular cameras, an image storage module and an image acquisition module used in a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments.

[0030] Figure 2 The present invention is a schematic diagram of the operation of a speckle array laser during the implementation of a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments.

[0031] Figure 3 The present invention is a schematic diagram of the areas captured by each camera during image acquisition in a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments.

[0032] Figure 4 A schematic diagram of speckle laser irradiation for identifying lifting bolts using a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments according to the present invention.

[0033] Figure 5 It is the top surface elliptical contour extracted when the elliptical contour is extracted by the multi-view speckle laser image measurement method for grabbing and positioning tunnel segments of the present invention.

[0034] Figure 6 It is a schematic diagram of the angles of different line segment columns when recognizing ellipses using a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments according to the present invention.

[0035] Figure 7The present invention is a schematic diagram of the principle of dividing the same area in a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments.

[0036] Figure 8 The diagram is a schematic diagram of a sub-area deformation principle based on a first-order shape function adopted in a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments according to the present invention.

[0037] Fig. 9 The present invention provides a flow chart of positioning lifting bolts for assembling shield construction segments in a multi-view speckle laser image measurement method for grabbing and positioning tunnel segments.

[0038] It should be noted that the dimensions of the segment assembly machine model diagram and the layout orientation diagram of each hardware in the visual analysis device mentioned in the drawings of this specification are not described in proportion and are only for the convenience of understanding.

[0039] In the figure, 11 is a grabbing plate at the end of the segment assembly machine, 21-a first speckle array laser, 22-a second speckle array laser, 23-a third speckle array laser, 24-a fourth speckle array laser, 31-a first monocular camera, 32-a second monocular camera, 33-a third monocular camera, 34-a fourth monocular camera, 35-a fifth monocular camera, 36-a sixth monocular camera, 41-an image storage module, and 51-an image processing module. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] A multi-view speckle laser image measurement method for tunnel segment grabbing and positioning, comprising the following steps:

[0042] S1. Arrange and install a visual analysis device on the end grabbing plate of the segment assembly machine of the shield machine, and the visual analysis device includes a speckle array laser and a monocular camera.

[0043] Furthermore, an image storage module and an image processing module are arranged and installed on the end grabbing plate of the segment assembly machine of the shield machine; six monocular cameras are all connected to the image storage module, and the image storage module is connected to the image processing module.

[0044] Furthermore, there are four groups of speckle array lasers and six monocular cameras. The four groups of speckle array lasers are used to emit speckle lasers, and the six monocular cameras are used to perform image acquisition tasks; the image storage module is used to perform the function of storing images; and the image processing module is used to process the acquired images. The four groups of speckle lasers have laser overlap, which reduces the image error caused by incomplete laser coverage; the six monocular cameras are evenly distributed on the grabbing plate at the end of the pipe segment assembly machine, and cross-overlap of image acquisition is performed to reduce the measurement error from the acquisition end.

[0045] Specifically, the four groups of speckle array lasers include a first speckle array laser 21, a second speckle array laser 22, a third speckle array laser 23, and a fourth speckle array laser 24; the six monocular cameras include a first monocular camera 31, a second monocular camera 32, a third monocular camera 33, a fourth monocular camera 34, a fifth monocular camera 35, and a sixth monocular camera 36.

[0046] Furthermore, if Figure 1 As shown, considering the orientation of the image acquisition and the "integration" principle of the hardware layout, the first speckle array laser 21 is fixed at the middle of the left edge of the end grabbing disk 11 of the segment assembly machine, the second speckle array laser 22 is fixed at the middle of the left front end of the end grabbing disk 11 of the segment assembly machine, the third speckle array laser 23 is fixed at the middle of the right front end of the end grabbing disk 11 of the segment assembly machine, and the fourth speckle array laser 24 is fixed at the middle of the right edge of the end grabbing disk 11 of the segment assembly machine. This arrangement can ensure that the speckle laser fully covers the segments to be grabbed and the lifting bolts, and reduce the error of the image source.

[0047] The first monocular camera 31 is fixed at the upper left corner of the left edge of the grabbing plate at the end of the segment assembly machine, the second monocular camera 32 is fixed at the lower right corner of the left edge of the grabbing plate 11 at the end of the segment assembly machine, the third monocular camera 33 is fixed at the right side of the left front end of the grabbing plate 11 at the end of the segment assembly machine, the fourth monocular camera 34 is fixed at the left side of the right front end of the grabbing plate 11 at the end of the segment assembly machine, the fifth monocular camera 35 is fixed at the upper right corner of the right edge of the grabbing plate 11 at the end of the segment assembly machine, and the sixth monocular camera 36 is fixed at the lower right corner of the right edge of the grabbing plate 11 at the end of the segment assembly machine. The monocular cameras are arranged to cover the grabbing direction of the segment assembly machine, so as to realize multi-view and all-round acquisition of images.

[0048] The image storage module 41 is fixed on the left side of the third monocular camera 33 , and the image processing module 51 is fixed on the right side of the fourth monocular camera 34 .

[0049] S2. When the translation mechanism of the segment assembly machine moves toward the segment to be grabbed in the direction of the shield tail, the speckle array laser is used to emit speckle laser, and the monocular camera is used to continuously collect speckle images of the segment to be grabbed and the lifting bolts.

[0050] like Figure 2 As shown, when four groups of speckle array lasers emit speckle lasers, the edges of the speckle lasers emitted by adjacent laser groups overlap, so as to ensure that the speckle lasers completely cover the surface of the pipe segment to be grasped.

[0051] The adjacent laser groups refer to: the first speckle array laser 21 and the second speckle array laser 22 , the second speckle array laser 22 and the third speckle array laser 23 , and the third speckle array laser 23 and the fourth speckle array laser 24 .

[0052] like Figure 3 As shown, when six monocular cameras are used to collect speckle images of the pipe segments to be grabbed and the lifting bolts, the six monocular cameras are respectively combined into five groups of binocular cameras for image collection: the first monocular camera 31, the second monocular camera 32; the second monocular camera 32, the third monocular camera 33; the third monocular camera 33, the fourth monocular camera 34; the fourth monocular camera 34, the fifth monocular camera 35; the fifth monocular camera 35, the sixth monocular camera 36. The speckle images collected by adjacent monocular camera groups overlap with each other to reduce measurement errors.

[0053] Among them, the first monocular camera 31 collects Figure 4 The second monocular camera 32 is capturing the area A shown in FIG. Figure 4 The B area shown in the figure overlaps with the A area captured by the first monocular camera 31, thereby achieving the effect of binocular camera image capture; the third monocular camera 33 captures Figure 4 The C area shown in the figure overlaps with the B area captured by the second monocular camera 32, forming a set of binocular camera captured images; the fourth monocular camera 34 captures Figure 4 The D area shown in the figure partially overlaps with the C area captured by the third monocular camera 33, forming a set of binocular camera captured images; the fifth monocular camera 35 captures Figure 4 The E area shown in the figure partially overlaps with the D area captured by the fourth monocular camera 34, forming a set of binocular camera captured images; the sixth monocular camera 36 captures Figure 4 The F area shown in FIG. 1 partially overlaps with the E area captured by the fifth monocular camera 35 to form a group of binocular camera captured images. Thus, six groups of monocular camera captured images to construct five groups of binocular camera image capture.

[0054] Furthermore, the specific distribution of AE area on the segment to be grasped is: A is the 0-1m area on the left edge of the segment to be grasped; B is the 0.8-1.2m area at the left edge of the segment to be grasped, and the 0-0.8m area at the front edge; C is the 0.6-1.8m area at the front edge of the segment to be grasped; D is the 1.6-2.8m area at the front edge of the segment to be grasped; E is the 2.6-3.4m area at the front edge of the segment to be grasped, and the 0.8-1.2m area at the right edge; F is the 0-1m area at the right edge of the segment to be grasped.

[0055] like Figure 4 As shown, when the speckle laser is irradiated onto the segment to be grasped and the lifting bolt, an irregular speckle laser intensity distribution will appear on the image due to the height difference between the upper surface of the segment to be grasped and the lifting bolt, and an unevenly distributed scattered spot cloud will appear on the image of the lifting bolt and the segment to be grasped, and the scattered spot cloud will present a granular structure pattern.

[0056] The acquired speckle image is input into the image storage module for temporary storage; the image storage module extracts the stored speckle image and inputs it into the image processing module.

[0057] S3. Use the improved YOLOV9 model to detect and identify the contours of the pipe segments and lifting bolts to be grasped in the speckle image, and extract the contours of the pipe segments and lifting bolts to be grasped.

[0058] Furthermore, the improved YOLOV9 model is a contour detection model based on the YOLOV9 model, and a self-attention mechanism is introduced in the backbone network layer of the YOLOV9 model to enhance the relationship between the input layers; combined with the LOG spot detection algorithm, a terminal output layer structure is added to the YOLOV9 model to match the target points generated for measurement; finally, the loss function CIoU Loss function in the original network structure of the YOLOV9 model is replaced with the EIoU Loss function to solve the problem of balancing difficult and easy samples not considered by the original loss function.

[0059] The improved YOLOV9 model performs comprehensive feature extraction on the images collected by five sets of binocular cameras. The RepNCSPELAN network layer structure is used to extract the features of the speckle image and generate a multi-scale feature map. The potential relationship between the five groups of images is enhanced through the self-attention mechanism, so that the key contour changes in the image can be identified. The measurement error caused by the imbalance of local difficult and easy samples in the prediction process is solved by replacing the EIoULoss loss function. The feature area is processed in combination with the added LOG output layer structure to obtain the final contour feature image.

[0060] The input speckle image is feature extracted through the basic YOLOV9 model to obtain the feature areas of the pipe segments to be grabbed and the lifting bolts; the self-attention mechanism is introduced in the backbone network layer to weight the feature areas extracted by the input layer network, so that the improved YOLOV9 model can pay attention to the key contours and local feature changes; combined with the LOG spot detection algorithm, the LOG output layer structure is added, so that the improved YOLOV9 model can directly output contour features; then the loss function CIoU Loss function in the original network structure of the YOLOV9 model is replaced with the EIoU Loss function to solve the problem of balancing difficult and easy samples not considered by the original loss function; finally, the feature area is processed through the introduced self-attention mechanism, combined with the added LOG output layer structure and the replaced EIoU Loss function to obtain the final contour feature image.

[0061] The YOLOV9 model incorporates the concept of programmable gradient information (PGI) to generate reliable gradients through auxiliary reversible branches; at the same time, it uses a lightweight GELAN architecture to optimize parameters, computational complexity, accuracy, and inference speed. The combined network of REP+CSP+ELAN is used as the feature extraction layer of the speckle image to realize the feature extraction of the speckle image. The GELAN lightweight network architecture is used to achieve high accuracy while maintaining the lightweight and high speed of the network. The RepNCSPELAN structure is the main branch of YOLOV9, which is responsible for feature extraction of the input speckle image, generating multi-scale feature maps, and then predicting the category and location information of the detected contour.

[0062] Relying on the YOLOV9 basic network structure, the self-attention mechanism is introduced into the backbone network structure layer to enhance the relationship between the input layers; the structure of the output layer is changed, and an output layer network is added so that it can output the contour segments of the pipe segments to be grasped and the lifting bolts; at the same time, the loss function is modified to solve the problem of balancing difficult and easy samples that is not considered in the original loss function.

[0063] The purpose of introducing the self-attention mechanism is to enhance the potential relationship between the input data of the YOLOV9 model, to weight the extracted contour features, so that the improved YOLOV9 model can focus on the key contours and local feature changes required in the speckle image, and to avoid the extremely poor model training effect. The self-attention network layer includes: matching the input data "query" in the current backbone network with the key; then using the "key" to represent other elements in the data to match the "query"; then constructing the "value", when the "key" matches the "query", using the relevant "value" to construct the output; finally calculating the similarity between the "key" and the "query", and calculating the contribution of the "value" to the output.

[0064] The purpose of adding the output layer network is to enable the network structure of the improved YOLOV9 model to output the contour feature points of the pipe segment and the lifting bolt to be detected. The added output layer mainly includes: introducing LOG spot detection as the output layer network structure added by the YOLOV9 model.

[0065] Define a LoG operator:

[0066]

[0067] Among them, G σ (x, y) is the value of the Gaussian function at position (x, y); x and y represent the coordinates of the edge pixels respectively; σ is the standard deviation and scale of the Gaussian function, which controls the degree of blur; exp is the power operation of the base e of the natural logarithm.

[0068]

[0069] in, represents the Laplace transform of the formula; G σ (x,y) represents the value of the Gaussian function at position (x,y); I(x,y) represents the coordinates of the edge pixel.

[0070] The LoG operator is used to apply Gaussian blur to the feature image input to the LOG output layer and then perform Laplace calculation.

[0071] When the scale σ of LoG matches the width of the spot (the spatial size of the spot in the image), the amplitude of LoG is maximized at the center of the spot. Then, LoG operators of different scales are convolved with the signal to find the maximum amplitude of LoG. The point that reaches the maximum (or minimum) value in both space and scale is the desired spot. The adjacent desired spots are connected in sequence to obtain the contour features existing in the entire speckle image, and the contours of the pipe segments and lifting bolts to be grabbed are extracted and output.

[0072] The above modification of the loss function in the network layer mainly includes: replacing the CIoU Loss function in the YOLOV9 model network layer with the EIoU Loss function. The CIoU Loss function increases the loss of the detection frame scale and increases the loss of length and width, so that the predicted frame will be more consistent with the real frame. Replacing it with the EIoU Loss function can replace the aspect ratio with the difference value of width and height, accelerate convergence, improve regression accuracy, and solve the problem of imbalance between difficult and easy samples.

[0073] Based on the principle of this method, the contour in the speckle image is extracted. The speckle images stored in the image storage module are processed in sequence, and the contour of the lifting bolt can be detected, thereby extracting the coordinates of the center point of the lifting bolt.

[0074] S4. The contour of the lifting bolt is processed by the ellipse detection algorithm of arc clustering to determine the coordinates of the center point of the lifting bolt on the pipe segment to be grasped. The coordinate transformation information of the image coordinates of the center point of the lifting bolt and the world coordinate system is obtained by subtracting the coordinate information of adjacent frames after coordinate conversion, thereby obtaining the relative position change of the center point of the lifting bolt in the speckle image of adjacent frames, which provides precise positioning and path guidance for the assembly mechanism to grasp the pipe segment.

[0075] The method for determining the center point coordinates of the lifting bolts on the segment to be grabbed is:

[0076] After extracting the contours of the segments to be grabbed and the lifting bolts, Figure 5 , Figure 6 As shown, arc clustering is performed on the ellipses on the lifting bolt contour to determine the complete ellipse.

[0077] Specifically, the angle formed between the extracted arc segments is used to determine whether the identified arc is complete. The start point P1 and the end point P2 of the identified arc segment are connected to the midpoint P3 respectively, and the angle θ is defined as the required calculation angle. Define the angle coverage threshold M ac , if the angle θ is lower than the angle coverage threshold M during measurement ac , then it is determined that the integrity of the arc recognition meets the set value; if the angle is greater than the angle coverage threshold M ac , the ellipse is considered incomplete and cannot be fitted.

[0078] M ac =m×θ max (3)

[0079] Where m represents the required angle coverage ratio, which is 0.8; θ max Indicates the maximum angle range of the recognized arc.

[0080] As attached Figure 7 As shown in the figure, the same area division is performed. The double endpoints of the arc to be clustered in the contour map of the pipe segment to be grabbed and the lifting bolt output by the improved YOLOV9 model are used as the starting point, and two rays are emitted, which are consistent with the direction of the arc tangent to form an effective search range S. The arcs searched within the effective search range S are the same ellipse. a2 and a3 are within the effective search range S, while a4 is not within the effective search range S. By limiting the search range of arc clustering, the detection efficiency is effectively improved.

[0081] Define two constraints to further determine whether the arc segment in the search range is the same ellipse as the arc to be clustered. Take the average position of the two points on the arc as the chord center, and judge the distance between the arc center and the chord center. The distance between the arc center and the chord center should be the maximum value. At this time, select the arc in the retention range, otherwise discard it; judge the distance of the center of the fitted ellipse based on the characteristics of the ellipse polar line, determine the tangent of the arc segment, and then define the distance between the arc segment and the ellipse center. At the same time, select the one with the largest diameter of each arc in the lifting bolt, and use its diameter as the judgment threshold D for comparison. If the distance between the arc segment and the ellipse center is less than the judgment threshold D, it is determined to be part of the ellipse, otherwise it is discarded.

[0082] The directed boundary sequence is extracted from the identified complete ellipse, and the ellipse parameters are fitted using the least square method.

[0083] Specifically, a threshold K is defined, and the shortest distance di between the edge pixel and the fitted ellipse boundary is calculated. Finally, all edge pixel points whose distance is less than the threshold K are calculated, and their proportion C in the total number of edge points is calculated. If C ≥ 65%, it is determined that the ellipse really exists.

[0084] K=∈×min (a,b) (4)

[0085] Where ∈ represents the ratio of the required minimum distance, which is 0.05; a and b represent the major and minor semi-axis of the ellipse, respectively.

[0086]

[0087] Among them, (x i ,y i ) represents the coordinates of an edge pixel point, (p,q) is the center coordinate of the fitted ellipse, θ represents the rotation angle of the fitted ellipse, r1 and r2 are the major axis and minor axis radii respectively.

[0088] Finally, the identified lifting bolt ellipses are screened to obtain the required lifting bolt top surface ellipse.

[0089] Because the shape of the lifting bolt is cylindrical at the head and bottom, multiple ellipses will be displayed when the monocular camera is not vertically recognized, and the target ellipse required for final positioning needs to be screened out. The required ellipse is the top ellipse of the lifting bolt at the top, so the Y value of the target ellipse should be the maximum value among the recognized ellipses. The set of detected lifting bolt ellipses is defined as E = {E1, E2, ..., E n}, n is the total number of ellipses detected in the feature image. The center coordinates of each ellipse are marked as (x ci ,y ci ), select the ellipse with the maximum Y coordinate as the required ellipse.

[0090]

[0091] Through the above detection and processing method, the lifting bolt contour can be detected to obtain the coordinates of the ellipse center point on the upper surface of the lifting bolt, that is, the calibrated coordinates of the lifting bolt.

[0092] After completing the above process, spatial coordinate conversion is performed. By extracting the position information of the lifting bolt center point in the image, coordinate conversion is performed to obtain the position of the lifting bolt point relative to the world coordinate system. The coordinate information of adjacent frames after coordinate conversion is subtracted to obtain the relative position change of the lifting bolt center point in the speckle image of adjacent frames, which provides precise positioning and path guidance for the assembly mechanism to grab the pipe segment.

[0093] like Figure 8 As shown in the figure, the calibrated coordinate image processed by the arc clustering algorithm is divided into several sub-areas. When the grabbing device moves, the extracted image will be deformed. A single sub-area is composed of a regular shape of (2M+1) pixels × (2M+1) pixels. The initial coordinates of the central pixel of the sub-area are set to P(x0, y0), and the distance coordinates of another point Q from point P are represented as (Δx, Δy) in the schematic diagram. When the segment assembly machine is translated or rotated, the extracted image will also be deformed, and the coordinates of the point Q′ after the deformation of the sub-area are expressed as a first-order function:

[0094]

[0095] Where x′ and y′ represent the position coordinates of point Q′ in the deformed image sub-area; u and v represent the displacement of the initial center point in the horizontal and vertical directions respectively; They are the gradients of v in the horizontal and vertical directions respectively; are the gradients of u in the horizontal and vertical directions respectively.

[0096] The coordinate transformation information of the image coordinates of the lifting bolt center point and the world coordinate system is obtained by subtracting the coordinate information of adjacent frames after coordinate conversion, thereby obtaining the relative position change of the lifting bolt center point in the speckle images of adjacent frames, providing precise positioning and path guidance for the assembly mechanism to grab the pipe segment.

[0097] In summary, if Fig. 9As shown, when the end of the segment assembly machine moves toward the lifting bolt, the present invention uses four groups of speckle array lasers to emit speckle lasers; five groups of binocular vision cameras are used to continuously collect images of the movement process; the speckle pattern is processed and analyzed using visual digital image technology to extract the contour features of the segment to be grasped and the lifting bolt; at the same time, the ellipse detection algorithm of arc clustering is used for processing to extract the coordinates of the center point of the lifting bolt; the coordinate transformation information of the image coordinates of the center point of the lifting bolt and the world coordinate system is obtained by sub-area deformation calculation; because the pattern on the image of each adjacent frame will be deformed, the coordinate information of adjacent frames after coordinate conversion is subtracted, so as to obtain the relative position change of the center point of the lifting bolt in the speckle image of the adjacent frames.

[0098] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-view speckle laser image measurement method for tunnel segment grabbing and positioning, characterized in that: The following steps are involved: S1. Arrange and install a visual analysis device on the end grabbing plate of the segment assembly machine of the shield machine, the visual analysis device including a speckle array laser and a monocular camera; S2. When the translation mechanism of the segment assembly machine moves toward the segment to be grabbed in the direction of the shield tail, the speckle array laser is used to emit speckle laser, and the monocular camera is used to continuously collect speckle images of the segment to be grabbed and the lifting bolts; S3. Use the improved YOLOV9 model to detect and identify the contours of the segments to be grabbed and the lifting bolts in the speckle image, and extract the contours of the segments to be grabbed and the lifting bolts; S4. The contour of the lifting bolt is processed by the ellipse detection algorithm of arc clustering to determine the coordinates of the center point of the lifting bolt on the pipe segment to be grasped. The coordinate transformation information of the image coordinates of the center point of the lifting bolt and the world coordinate system is obtained by subtracting the coordinate information of adjacent frames after coordinate conversion, thereby obtaining the relative position change of the center point of the lifting bolt in the speckle image of adjacent frames, which provides precise positioning and path guidance for the assembly mechanism to grasp the pipe segment.

2. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 1 is characterized in that: The speckle array lasers are provided in four groups, and the four groups of speckle array lasers include a first speckle array laser (21), a second speckle array laser (22), a third speckle array laser (23), and a fourth speckle array laser (24); the first speckle array laser (21) is fixed to the middle of the left edge of a gripping plate (11) at the end of the pipe segment assembly machine, the second speckle array laser (22) is fixed to the middle of the left front end of the gripping plate (11) at the end of the pipe segment assembly machine, the third speckle array laser (23) is fixed to the middle of the right front end of the gripping plate (11) at the end of the pipe segment assembly machine, and the fourth speckle array laser (24) is fixed to the middle of the right edge of the gripping plate (11) at the end of the pipe segment assembly machine.

3. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 2 is characterized in that: There are six monocular cameras, which include a first monocular camera (31), a second monocular camera (32), a third monocular camera (33), a fourth monocular camera (34), a fifth monocular camera (35), and a sixth monocular camera (36); the first monocular camera (31) is fixed to the upper left corner of the left edge of the grabbing tray (11) at the end of the pipe segment assembly machine, the second monocular camera (32) is fixed to the lower right corner of the left edge of the grabbing tray (11) at the end of the pipe segment assembly machine, the third monocular camera (33) is fixed to the right side of the left front end of the grabbing tray (11) at the end of the pipe segment assembly machine, the fourth monocular camera (34) is fixed to the left side of the right front end of the grabbing tray (11) at the end of the pipe segment assembly machine, the fifth monocular camera (35) is fixed to the upper right corner of the right edge of the grabbing tray (11) at the end of the pipe segment assembly machine, and the sixth monocular camera (36) is fixed to the lower right corner of the right edge of the grabbing tray (11) at the end of the pipe segment assembly machine.

4. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 3 is characterized in that: When the monocular cameras are used to collect speckle images of the pipe segments to be grasped and the lifting bolts, the six monocular cameras are respectively combined into five groups of binocular cameras for image collection: a first monocular camera (31) and a second monocular camera (32); a second monocular camera (32) and a third monocular camera (33); a third monocular camera (33) and a fourth monocular camera (34); a fourth monocular camera (34) and a fifth monocular camera (35); and a fifth monocular camera (35) and a sixth monocular camera (36), and the speckle images collected by adjacent monocular camera groups overlap with each other.

5. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 4 is characterized in that: The first monocular camera (31) collects speckle images of the 0-1 m area on the left edge of the pipe segment to be grasped, the second monocular camera (32) collects speckle images of the 0.8-1.2 m area on the left edge of the pipe segment to be grasped and the 0-0.8 m area on the front edge, the third monocular camera (33) collects speckle images of the 0.6-1.8 m area on the front edge of the pipe segment to be grasped, the fourth monocular camera (34) collects speckle images of the 1.6-2.8 m area on the front edge of the pipe segment to be grasped, the fifth monocular camera (35) collects speckle images of the 2.6-3.4 m area on the front edge of the pipe segment to be grasped and the 0.8-1.2 m area on the right edge, and the sixth monocular camera (36) collects speckle images of the 0-1 m area on the right edge of the pipe segment to be grasped.

6. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 1 or 5, characterized in that: The improved YOLOV9 model is obtained by introducing a self-attention mechanism into the backbone network layer of the YOLOV9 model; adding a terminal output layer network LOG output layer combined with LOG spot detection into the YOLOV9 model; and replacing the loss function CIoU Loss function in the original network structure of the YOLOV9 model with the EIoU Loss function.

7. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 6 is characterized in that: The method of outputting the outline of the pipe segment and the lifting bolt to be grabbed by the LOG output layer is: Gaussian blur is applied to the feature image input to the LOG output layer by the LoG operator, and then Laplace calculation is performed. The LoG operator is: When the scale σ of LoG matches the width of the spot, the amplitude of LoG reaches the maximum at the center of the spot; then the LoG operators of different scales are used to convolve with the signal to find the maximum amplitude of LoG; the point that reaches the maximum or minimum value in space and scale is the desired spot, and the adjacent desired spots are connected in sequence to obtain the contour features existing in the entire speckle image, and the contours of the pipe segment and lifting bolt to be grabbed are extracted; where G σ (x, y) is the value of the Gaussian function at position (x, y), x and y represent the coordinates of the edge pixel points, σ ​​is the standard deviation and scale of the Gaussian function, exp is the power operation of the base e of the natural logarithm, Denotes Laplace transform, G σ (x,y) represents the value of the Gaussian function at position (x,y), and I(x,y) represents the coordinates of the edge pixel.

8. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 7 is characterized in that: The method for determining the center point coordinates of the lifting bolt on the pipe segment to be grasped is: arc clustering of the ellipses on the lifting bolt contour to determine the complete ellipse; extracting a directed boundary sequence for the identified complete ellipse, and fitting the ellipse parameters using the least squares method; screening the identified lifting bolt ellipses to obtain the required lifting bolt top surface ellipse, and the center point coordinates of the lifting bolt top surface ellipse are the center point coordinates of the lifting bolt.

9. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 8, characterized in that: The method for clustering arcs of ellipses on the contour of lifting bolts is as follows: taking the double endpoints of the arcs to be clustered in the contour map of the pipe segment to be grasped and the lifting bolt output by the improved YOLOV9 model as the starting point, emitting two rays in the same direction as the tangent direction of the arc to form an effective search range, and the arcs searched within the effective search range are the same ellipse; and taking the average position of the two points on the arc as the chord center, judging the distance between the arc center and the chord center, the arc center and the chord center should be the maximum value, and at this time, the arc retained in the effective search range is selected, otherwise it is discarded; judging the distance of the center of the fitted ellipse is judged based on the characteristics of the ellipse polar line, determining the tangent of the arc segment, and then defining the distance between the arc segment and the center of the ellipse, and selecting the diameter of the largest arc in the lifting bolt as the judgment threshold for comparison, if the distance between the arc segment and the center of the ellipse is less than the judgment threshold, it is determined to be part of the ellipse, otherwise it is discarded.

10. The multi-view speckle laser image measurement method for tunnel segment grabbing and positioning according to claim 2, 3 or 9, characterized in that: An image storage module (41) and an image processing module (51) are also arranged and installed on the end grabbing disk of the segment assembly machine of the shield machine. The image storage module (41) is fixed on the left side of the third monocular camera (33), and the image processing module (51) is fixed on the right side of the fourth monocular camera (34); the first monocular camera (31), the second monocular camera (32), the third monocular camera (33), the fourth monocular camera (34), the fifth monocular camera (35), and the sixth monocular camera (36) are all connected to the image storage module (41), and the image storage module (41) is connected to the image processing module (51).

Images