An automated method and system for transporting tunnel segments

Abstract

This invention discloses an automatic segment hoisting method and system, solving the technical problems of low positioning accuracy and limited applicability of existing segment hoisting methods. This invention utilizes machine vision to quickly identify segment stacks and their corresponding segment types, segment positioning marks, and segment transport vehicle placement marks; it improves positioning accuracy by constructing a binocular camera model and segment structural parameters to correct for crane tilt; and it enhances computational positioning accuracy by using a camera transformation matrix to convert feature point coordinates to the same coordinate system for calculating corresponding correction deviations.

Description

Technical Field

[0001] This invention relates to the field of automated tunnel segment hoisting technology, and in particular to an automated tunnel segment hoisting method and system. Background Technology

[0002] During tunnel construction, tunnel segments need to be transported from the ground to the tunnel in stacks. Construction workers must locate the required sequence of segments from the stacks and then control a crane to lift them one by one to the target location. During lifting, workers need to align the crane with the segments before the crane can grab them, a process that consumes a significant amount of time. Furthermore, the grabbing accuracy is affected by the workers' operating habits. If the grabbing accuracy cannot be guaranteed, there is a possibility that the segments may fall off the crane during transport. Therefore, using an automated system to replace human judgment and achieve intelligent crane grabbing can not only improve work efficiency but also ensure construction safety and save economic costs.

[0003] In recent years, with the development of technology, research on automated tunnel segment lifting has gradually emerged. Many methods employ LiDAR to locate tunnel segments by identifying their geometric contours, or image processing methods to identify segments through threshold segmentation. However, these methods suffer from slow processing speeds, recognition rates that decrease with changes in the working environment, and inconsistent positioning accuracy. Therefore, it is essential to utilize machine vision methods for tunnel segment identification and handling.

[0004] For example, Chinese invention patent application CN 114439519 A, published on May 6, 2022, discloses an automatic segment hoisting system, an automatic segment hoisting method, and a tunnel boring machine. The automatic segment hoisting system includes a segment hoisting device, a contour measuring device, a vision device, and a transfer platform. The contour measuring device is fixed to the segment hoisting device and configured to scan the contours of multiple segments to be hoisted in a first area to obtain their contour information. The vision device is fixed to the segment hoisting device and configured to acquire image information of the segment currently to be hoisted. This automatic segment hoisting system achieves automatic identification of the segments to be hoisted through the contour measuring device and the vision device, enabling the automatic hoisting of multiple segments from the first area to a second area.

[0005] Chinese invention patent application CN 109826648 A, published on May 31, 2019, discloses a binocular segment assembly recognition system and its assembly recognition method. The system calculates the position and three-dimensional coordinates of the segments and bolt holes on the segments using a first binocular recognition module and a second binocular recognition module. This provides guidance information for the assembly machine to grasp and assemble the segments, thus completing the segment grasping and assembly task with high efficiency and accuracy.

[0006] Although both of the above patent applications disclose a method for visual recognition, positioning, and gripping of pipe segments, when calculating the positioning deviation, they rely on the prior conditions that the positioning hole and the grouting hole of the pipe segment to be gripped are on the same straight line to determine the position of the positioning hole and indirectly obtain the positioning deviation. They depend on the geometric constraints between the holes and cannot directly calculate the motion deviation of the positioning hole relative to the gripping device. Therefore, the positioning accuracy is low and the applicable range is small. Summary of the Invention

[0007] To address the shortcomings in the aforementioned background technology, this invention proposes an automatic segment hoisting method and system, which solves the technical problems of low positioning accuracy and limited applicability of existing segment hoisting methods.

[0008] The technical solution of this application is implemented as follows: an automatic segment hoisting method, comprising the following steps:

[0009] A. Establish a segment type database based on the geometric characteristics of different segment categories. Identify segment types using images captured by a binocular camera on the crane and a segment type identification network model. Establish a segment capture order database.

[0010] B. Calculate the height of the tunnel segment in the camera coordinate system using the images acquired by the binocular camera, correct the current position of the crane based on the height value and the corresponding tunnel segment structural parameters, and record the three-dimensional coordinate value of the crane after correction.

[0011] C. Based on the corrected crane position, the image location area of ​​the segment positioning hole is obtained by combining the segment image acquired by the binocular camera with the segment type identification network model. The coordinates of the image location area are transformed into three-dimensional coordinates in the camera coordinate system. The feature points in the image location area are evaluated by the feature evaluation function to obtain the positioning coordinates of the segment positioning hole.

[0012] D. Calculate the deviation between the coordinate position of the positioning hole in the camera coordinate system and the coordinate position of the crane positioning pin in the camera coordinate system, correct the current posture of the crane, and control the crane to complete the engagement of the crane positioning pin with the segment positioning hole.

[0013] Furthermore, in step A, a segment type database is established based on the geometric features of different types of segments. The binocular camera on the crane scans the uppermost segment in different segment stacks, and the collected segment images are input into the segment type identification network model to obtain the corresponding segment type. A segment grabbing order library is established based on the identified segment type and transmitted to the PLC central control system.

[0014] Further, in step B, based on the segment grabbing sequence library, the PLC central control system controls the crane to move above the target segment. The binocular camera on the crane acquires images, calculates the parallax of the common area of ​​the left and right cameras, calculates the height value of the segment in the camera coordinate system, and corrects the current position of the crane according to the calculated height value and the corresponding segment structure parameters. If the current position of the crane exceeds the allowable tilt range, the PLC central control system corrects the position of the crane and controls the crane to be lowered to the corrected precise positioning height, and records the corrected three-dimensional coordinate value of the crane.

[0015] Further, in step C, based on the corrected crane position, the left and right cameras acquire images of the tunnel segment, and input the acquired images of the tunnel segment into the target recognition network to obtain the image position areas of the left and right positioning holes on the tunnel segment. According to the image coordinates of the positioning holes and the distance between the binocular camera and the tunnel segment obtained from the PLC central control system, the coordinates of the image position areas are transformed into three-dimensional coordinates in the camera coordinate system. The feature points in the image position areas are evaluated by the feature evaluation function to obtain the positioning coordinates of the positioning holes.

[0016] Further, in step D, the deviation between the coordinate position of the positioning hole in the camera coordinate system and the coordinate position of the crane positioning pin in the camera coordinate system is calculated and transmitted to the PLC central control system. The PLC central control system sends control commands to the crane to correct the current posture of the crane and control the crane to complete the engagement of the crane positioning pin with the segment positioning hole. After the crane reaches the lowering height, the crane grabs the segment and transports it to the predetermined position.

[0017] Further, in steps C and D, the coordinates of the segment positioning hole in the camera coordinate system are calculated using the camera model. The initial center coordinates of the positioning hole are calculated based on the acquired feature point set. The initial center coordinates of the positioning hole and the feature point set are input into a feature evaluation function to evaluate the feature points and obtain the feature points with the maximum values. The precise coordinates of the positioning hole are calculated based on the acquired feature point set. The coordinates of one camera are transformed to the coordinate system of the other camera using transformation matrices from the left and right cameras. The deviation includes the positional deviation of the identified segment positioning target relative to the target positioning [d]. x d y ] and rotational deviation [d r The calculation formula is as follows: dr = cos(V) 吊机位 V 标定位 ), dx, dy=f(V 吊机位 V 标定位 ).

[0018] Furthermore, the predetermined position is the segment placement position on the segment transport vehicle. After the crane grabs the segment and before lifting it to the segment placement position, the segment transport vehicle positioning camera captures an image of the current segment transport vehicle, identifies the segment placement position marker on the segment transport vehicle, and calculates the planar coordinates P of the segment transport vehicle placement marker in the camera coordinate system. S [X, Y], transform the position coordinates of the crane and the segment transport vehicle to the same coordinate system, and calculate the position of the crane and the current rotational deviation of the segment transport vehicle [d]. r ] and horizontal deviation [d x d y The deviation is transmitted to the PLC central control system to adjust the crane's planar deviation. x d y ] and rotational deviation [d r After the crane is aligned, the PLC central control system controls the crane to lift the tunnel segment to the segment placement position, and then the crane returns to the ready-to-work position.

[0019] Furthermore, the feature evaluation function is:

[0020]

[0021] K1 and K2 are adjustable parameters that can be adjusted according to different scenarios to balance distance evaluation and data distribution evaluation; [X] i Y i For each feature point, [X] m Y m [a, b] represents the geometric center point of the identified image location region, [a, b] represents the image coordinates of the current feature point, g represents the image grayscale value of the current point, and n is the current feature point [X]. i Y i Relative to the geometric center point [X] m Y m The total number of feature points in the distance distribution, where N is the total number of feature points.

[0022] Furthermore, the image location area is a rectangular area containing the target. The image location area is divided into three sub-regions according to the 120-degree arithmetic progression. The distance and height of the binocular camera from the tube segment are obtained from the PLC central control system. The image coordinates of the three sub-regions are converted into three-dimensional coordinates in the camera coordinate system. Each feature point is evaluated by the feature evaluation function D to obtain the set of positioning coordinates of the target positioning hole. The center coordinates are fitted based on the feature points of the three sub-regions, which are the coordinates of the target positioning hole.

[0023] Furthermore, the network model for identifying the type of pipe segment is designed as a target recognition network structure Model1, using a convolutional network as the feature extraction framework. When identifying the type of pipe segment, the features extracted by the network are used as input, and a softmax classifier is used to classify the features to obtain the type of the current pipe segment. There is no need to perform position regression on the pipe segment markers; only the target category needs to be identified.

[0024] Furthermore, during the segment grabbing and positioning and the segment placement positioning, since it is necessary to identify the image coordinates of the segment hole and the segment transport vehicle placement position, on the basis of Model 1, the target image position deviation is added to the loss function to construct Model 2. The extracted features are input into the target classifier and the position regressor to achieve target object classification and position regression.

[0025] As a parallel technical solution that uses a binocular camera to directly identify positioning holes, a lidar is installed on the crane base to obtain the point cloud of the current scene, and the positioning hole is identified by the geometric features of the segment positioning hole; or markers are placed around the positioning hole, and the positioning hole is indirectly identified by identifying the corresponding markers.

[0026] An automated tunnel segment hoisting system includes a crane, a binocular camera, and a positioning camera for a tunnel segment transport vehicle. All three components are connected to a PLC central control system. The PLC central control system controls the binocular camera to acquire images and identifies the target tunnel segment type from a tunnel segment type database using a tunnel segment type recognition network model. The system also controls the binocular camera to acquire images and obtains the positioning coordinates of the tunnel segment positioning holes using a feature evaluation function. The PLC central control system calculates the deviation between the positioning coordinates of the positioning holes and the coordinates of the crane's positioning pins. Based on this deviation, the PLC central control system corrects the crane's posture to achieve the engagement between the crane's positioning pins and the tunnel segment positioning holes.

[0027] Furthermore, the binocular camera is installed at the bottom of the crane, with the two cameras in the binocular camera being diagonally distributed, and the segment positioning device of the two cameras being located on the centerline of the crane.

[0028] Furthermore, the two cameras of the binocular camera are pinhole cameras. The pinhole cameras are mounted on a crane by a protective device. The protective device includes a protective cover disposed around the pinhole camera. An annular light source is disposed between the inner ring of the protective cover and the outer ring of the pinhole camera. A gas flushing mechanism is disposed on the protective cover facing the pinhole camera and the annular light source.

[0029] Furthermore, the gas flushing mechanism is adjustable in angle relative to the pinhole camera and there are two of them, which are symmetrically arranged on both sides of the protective cover about the pinhole camera.

[0030] Furthermore, each gas flushing mechanism includes a nozzle connected to a gas source device. The nozzles of the two gas flushing mechanisms are mounted coaxially and at the same height. The nozzles are tilted upwards to blow airflow toward the pinhole camera and the ring light source.

[0031] Furthermore, the crane's crane ropes are equipped with absolute encoders to monitor the rope length and determine whether the crane is tilted; or laser rangefinders are installed around the crane to obtain azimuth coordinates, calculate the crane's current pose relative to the tunnel segment, and determine whether it is tilted.

[0032] Compared with the prior art in the background section, this invention can quickly identify the segment stacks and their corresponding segment types, segment positioning marks, and segment transport vehicle placement marks through machine vision methods; it improves positioning accuracy by constructing a binocular camera model and segment structure parameters to correct whether the current crane is tilted; it improves calculation positioning accuracy by using a camera transformation matrix to convert the feature point coordinates into the same coordinate system to calculate the corresponding correction deviation; and it enhances the calculation positioning accuracy by using gas to clean the camera lens, which helps extend the service life of the camera equipment and effectively avoids the lens being blocked by liquid and dust when using liquid cleaning in the dusty environment of the tunnel. Therefore, the segment hoisting system and method established by machine vision and camera model effectively improves tunnel construction efficiency, has higher positioning calculation accuracy, stronger anti-interference ability, and lower economic cost.

[0033] The method proposed in this invention does not rely on the geometric constraints between the segment positioning holes, but directly calculates the motion deviation of the segment positioning holes relative to the crane. Furthermore, when calculating the center coordinates of the segment positioning holes, this invention proposes an evaluation function based on a combination of distance and probability distribution to obtain reliable and accurate center coordinates. Additionally, during target identification, based on the special characteristics of the segments—including hoisting positioning holes, holes for assembling the lifting head, and grouting holes with similar features—the unique spatial geometric distribution characteristics among the various holes in the segments are used as the judgment criterion to identify the positioning holes and improve identification accuracy, thus avoiding misjudgment by the identification model. Here, the feature geometric distribution refers only to the distribution of relative spatial positions and is not constrained by absolute linearity or angular conditions. In comparison, the method adopted in this invention has a wider range of applications, is not affected by absolute geometric constraint errors between the segment's own markers, and has higher positioning accuracy.

[0034] The segment type identification method in this invention differs from existing methods. This invention does not rely on placing QR codes on the segments and analyzes the actual construction conditions on site, as placing QR codes on every segment is inefficient. When identifying segments, this invention establishes a backend database based on the geometric features of different segment categories. By identifying the segment model characters present at the factory, the segment type is identified, and compared with the backend database to assign characteristic parameters to the current segment. Simultaneously, this invention proposes a method to invert the crane posture using the acquired segment point cloud, verifying whether the crane is tilted and providing feedback correction. During segment placement, camera calibration technology is used to establish the spatial positioning relationship between the segment transport vehicle placement position and the crane, achieving one-step corrective placement. A PLC serves as the control center of the entire system, coordinating various system submodules and ensuring efficient collaboration and interaction between them. High-pressure airflow is used to remove dust adhering to the lens, maintaining a clear lens field of view.

[0035] In summary, according to the present invention, machine vision technology can accurately identify the type of tunnel segments and establish the hoisting sequence, achieving high-precision positioning for segment grabbing and placement. The PLC central control system realizes the end-to-end process of segment placement from the segment stack to the segment placement without human intervention. It has a wide range of applications, high positioning accuracy, and can improve tunnel construction efficiency and reduce economic costs. Attached Figure Description

[0036] To more clearly illustrate the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a diagram showing the positional relationship of the binocular cameras at the bottom of the crane;

[0038] Figure 2 A bottom view of the camera and protective device after assembly;

[0039] Figure 3 This is a block diagram illustrating the principle of the method of the present invention;

[0040] Figure 4 To identify the status diagram of the segment stacking;

[0041] Figure 5 A map showing the relative positions of selected feature points within a region of the image.

[0042] Figure 6 A diagram showing the relationship between the target and the marker after the coordinate system is unified by the transformation matrix of the binocular camera.

[0043] Figure 7 for Figure 2 Top view;

[0044] Numbering on the map:

[0045] 1. Camera; 2. Segment positioning device; 3. Protective cover; 4. Ring light source; 5. Air source device; 6. Nozzle; 7. Crane; 8. Segment. Detailed Implementation

[0046] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0047] An automated segment hoisting method includes the following steps:

[0048] A. Establish a segment type database based on the geometric characteristics of different segment categories. Identify segment types using images captured by a binocular camera on the crane and a segment type identification network model. Establish a segment capture order database.

[0049] B. Calculate the height of the tunnel segment in the camera coordinate system using the images acquired by the binocular camera, correct the current position of the crane based on the height value and the corresponding tunnel segment structural parameters, and record the three-dimensional coordinate value of the crane after correction.

[0050] C. Based on the corrected crane position, the image location area of ​​the segment positioning hole is obtained by combining the segment image acquired by the binocular camera with the segment type identification network model. The coordinates of the image location area are transformed into three-dimensional coordinates in the camera coordinate system. The feature points in the image location area are evaluated by the feature evaluation function to obtain the positioning coordinates of the segment positioning hole.

[0051] D. Calculate the deviation between the coordinate position of the positioning hole in the camera coordinate system and the coordinate position of the crane positioning pin in the camera coordinate system, correct the current posture of the crane, and control the crane to complete the engagement of the crane positioning pin with the segment positioning hole.

[0052] In a preferred embodiment of the automated tunnel segment hoisting method, in step A, a tunnel segment type database is established based on the geometric characteristics of different types of tunnel segments. For example... Figure 3As shown, the binocular camera on the crane 7 scans the uppermost segment 8 in different segment stacks, inputs the collected segment image into the segment type identification network model, identifies the factory mark on the segment, and then obtains the corresponding segment type. Based on the identified segment type, a segment capture sequence library [[3.1, 3.2, 3.3], [1.1, 1.2, 1.3], [2.1, 2.2, 2.3]...] is established and transmitted to the PLC central control system.

[0053] In a preferred embodiment of the automated tunnel segment hoisting method, in step B, based on the tunnel segment grabbing sequence library, the PLC central control system controls the crane to move above the target tunnel segment. The binocular cameras on the crane acquire images, calculate the parallax of the common area between the left and right cameras, calculate the height of the tunnel segment in the camera coordinate system, and correct the crane's current position based on the calculated height and corresponding tunnel segment structural parameters. If the crane's current position exceeds the allowable tilt range, the PLC central control system corrects the crane's position and lowers it to the corrected precise positioning height, recording the corrected three-dimensional coordinates.

[0054] In a preferred embodiment of the automated tunnel segment hoisting method, in step C, based on the corrected crane position, two cameras (left and right) acquire images of the tunnel segment. These images are then input into a target recognition network to obtain the image location regions of the two positioning holes on the tunnel segment. Preferably, these image location regions are rectangular areas encompassing the target. Based on the image coordinates of the positioning holes and the distance between the binocular cameras and the tunnel segment obtained from the PLC central control system, the coordinates of the image location regions are transformed into three-dimensional coordinates in the camera coordinate system. Feature points in the image location regions are evaluated using a feature evaluation function to obtain the positioning coordinates of the positioning holes.

[0055] Furthermore, the feature evaluation function is:

[0056]

[0057] K1 and K2 are adjustable parameters that can be adjusted according to different scenarios to balance distance evaluation and data distribution evaluation; [X] i Y i For each feature point, [X] m Y m [a, b] represents the geometric center point of the identified image location region, [a, b] represents the image coordinates of the current feature point, g represents the image grayscale value of the current point, and n is the current feature point [X]. i Y i Relative to the geometric center point [X] m Y m The total number of feature points in the distance distribution, where N is the total number of feature points.

[0058] The image location area is a rectangular area containing the target. The image location area is divided into three sub-regions according to the 120-degree arithmetic progression. The distance and height of the binocular camera from the tube segment are obtained from the PLC central control system. The image coordinates of the three sub-regions are converted into three-dimensional coordinates in the camera coordinate system. Each feature point is evaluated by the feature evaluation function D to obtain the set of positioning coordinates of the target positioning hole. The center coordinates are fitted based on the feature points of the three sub-regions, which are the coordinates of the target positioning hole.

[0059] In a preferred embodiment of the automatic segment hoisting method, in step D, the deviation between the coordinate position of the positioning hole in the camera coordinate system and the coordinate position of the crane positioning pin in the camera coordinate system is calculated and transmitted to the PLC central control system. The PLC central control system sends control commands to the crane to correct the current posture of the crane and control the crane to complete the engagement of the crane positioning pin with the segment positioning hole. After the crane is lowered to the lowering height, the crane grabs the segment and transports it to the predetermined position.

[0060] As a preferred embodiment of the automatic segment hoisting method, the principle block diagram of the present invention is as follows: Figure 3 As shown, the predetermined position is the placement position of the tunnel segment on the tunnel segment transport vehicle. After the crane grabs the tunnel segment and before lifting it to the placement position, the positioning camera of the tunnel segment transport vehicle captures an image of the current tunnel segment transport vehicle, identifies the tunnel segment placement position mark on the tunnel segment transport vehicle, and calculates the planar coordinates P of the placement mark position in the camera coordinate system. S [X, Y], transform the position coordinates of the crane and the segment transport vehicle to the same coordinate system, and calculate the position of the crane and the current rotational deviation of the segment transport vehicle [d]. r ] and horizontal deviation [d x d y The deviation is transmitted to the PLC central control system to adjust the crane's planar deviation. x d y ] and rotational deviation [d r After the crane is aligned, the PLC central control system controls the crane to lift the tunnel segment to the segment placement position, and then the crane returns to the ready-to-work position.

[0061] In a preferred embodiment of the automatic segment hoisting method, in steps C and D, the coordinates of the segment positioning hole in the camera coordinate system are calculated using a camera model. The initial center coordinates of the positioning hole are calculated based on the acquired feature point set. The initial center coordinates of the positioning hole and the feature point set are input into a feature evaluation function to evaluate the feature points and obtain the feature point with the maximum value. The precise coordinates of the positioning hole are calculated based on the acquired feature point set. The coordinates of one camera are transformed to the coordinate system of the other camera using transformation matrices from the left and right cameras. The deviation includes the positional deviation of the identified segment positioning target relative to the target positioning [d]. x d y ] and rotational deviation [d r The calculation formula is as follows: dr = cos(V) 吊机位 V 标定位 ), dx, dy=f(V 吊机位 V 标定位 ).

[0062] Furthermore, the network model for identifying the type of pipe segment is designed as a target recognition network structure Model1, using a convolutional network as the feature extraction framework. When identifying the type of pipe segment, the features extracted by the network are used as input, and a softmax classifier is used to classify the features to obtain the type of the current pipe segment. There is no need to perform position regression on the pipe segment markers; only the target category needs to be identified.

[0063] Furthermore, during the segment grabbing and positioning and the segment placement positioning, since it is necessary to identify the image coordinates of the segment hole and the segment transport vehicle placement position, on the basis of Model 1, the target image position deviation is added to the loss function to construct Model 2. The extracted features are input into the target classifier and the position regressor to achieve target object classification and position regression.

[0064] As another implementation of the automatic segment hoisting method, and as a parallel technical solution that uses a binocular camera to directly identify the positioning hole, a lidar is installed on the crane base to obtain the point cloud of the current scene, and the positioning hole is identified by the geometric features of the segment positioning hole; or markers are placed around the positioning hole, and the positioning hole is indirectly identified by identifying the corresponding markers.

[0065] As a preferred embodiment of the automated tunnel segment hoisting method, the specific construction method of the tunnel segment type network model is as follows:

[0066] S1: Segment identification primarily aims to determine the current segment type. Therefore, a target recognition network structure, Model 1, is designed, using a convolutional network as the feature extraction framework. The features extracted by the network are used as input, and a softmax classifier is directly employed to classify the features and obtain the category to which the current segment belongs. Since position regression of the segment markers is not required, only target category identification is needed, reducing the number of learning parameters while maintaining the recognition rate.

[0067] S2: During the segment grabbing and positioning and segment placement positioning, since it is necessary to identify the image coordinates of the segment hole and the segment transport vehicle placement position, Model 2 is constructed by adding the target image position deviation to the loss function based on Model 1. The extracted features are input into the target classifier and the position regressor to achieve target object classification and position regression.

[0068] As a preferred embodiment of the automated tunnel segment lifting method, the steps for tunnel segment identification are as follows:

[0069] S1: The crane descends to the scanning height;

[0070] S2: Turn on the camera and sample images of different stacks of pipe segments, then input the images into the target recognition network Model1;

[0071] S3: Establish the hoisting sequence based on the different segment stack types output by the identification network Model1.

[0072] As a preferred embodiment of the automated tunnel segment hoisting method, the tunnel segment gripping and positioning steps are as follows:

[0073] S1: As Figure 4 As shown, crane 1 descends to the working height, and the binocular cameras take pictures of segment 3. The height information of the segment relative to the crane is calculated based on the parallax of the common area. A local point cloud library is established using the local height information. The local point cloud is divided into four parts by planar angles, and the three-dimensional geometric center of each part is calculated to obtain four spatial feature points. Based on the four spatial feature points, the pose of the segment in the camera coordinate system and the corresponding segment structural parameters are established as follows: Figure 5 As shown, it is determined whether the calculated pose is within the allowable range. If it exceeds the range, the tilt deviation generated by the crane is corrected based on the calculated local height information.

[0074] S2: The crane moves to the working height position, and the images captured by the left and right cameras are input into Model2. The model identifies the segment positioning holes and image coordinates in the images and divides them into three sub-regions. The coordinates of the segment positioning holes in the camera coordinate system are calculated using the camera model. The initial center coordinates of the positioning holes are calculated based on the acquired feature point set. The initial center coordinates of the positioning holes and the feature points of each sub-region are input into the established D function. The feature points are evaluated, and the feature point with the maximum value is obtained. The precise coordinates of the positioning holes are calculated based on the acquired feature point set. The camera coordinates of the right hole of the segment are transformed to the left camera coordinate system using the left and right camera transformation matrix. Figure 6 As shown, the positional and rotational deviations of the identified segment positioning target 1 relative to the positioning target 2 are calculated using the following formulas;

[0075] dr=cos(V 吊机位 V 标定位 )

[0076] dx, dy = f(V) 吊机位 V 标定位 )

[0077] S3: The deviation is transmitted to the industrial control computer, which controls the crane to adjust the deviation and lower the crane to the positioning part. The lowering height is determined by the displacement sensor of the top rod on the crane. After the crane reaches the length of the top rod, the crane's grippers close.

[0078] As a preferred embodiment of the automated tunnel segment hoisting method, the tunnel segment placement and positioning steps are as follows:

[0079] S1: The positioning camera of the segment transport vehicle is turned on to take a picture and input into Model2 to obtain the image coordinates of the placement position on the segment transport vehicle;

[0080] S2: Convert the flag coordinates to camera coordinates using the camera model;

[0081] S3: Calculate the positional deviation of the placement position relative to the crane and transmit it to the industrial control computer to adjust the crane;

[0082] S4: The crane is lowered onto the segment transport vehicle.

[0083] An automated tunnel segment hoisting system includes a crane, a binocular camera, and a positioning camera for a tunnel segment transport vehicle. All three components are connected to a PLC central control system. The PLC central control system controls the binocular camera to acquire images and identifies the target tunnel segment type from a tunnel segment type database using a tunnel segment type recognition network model. The system also controls the binocular camera to acquire images and obtains the positioning coordinates of the tunnel segment positioning holes using a feature evaluation function. The PLC central control system calculates the deviation between the positioning coordinates of the positioning holes and the coordinates of the crane's positioning pins. Based on this deviation, the PLC central control system corrects the crane's posture to achieve the engagement between the crane's positioning pins and the tunnel segment positioning holes.

[0084] Specifically, this includes a segment type identification system that scans and identifies stacked segments, builds a segment type database, and establishes a hoisting sequence list that is transmitted to the PLC central control system.

[0085] As a preferred embodiment of an automated tunnel segment hoisting system, it includes a tunnel segment gripping and positioning system that calibrates the coordinate positions of positioning pins in an image. For example... Figure 1 As shown, the binocular camera is installed at the bottom of the crane 7, with the two cameras 1 arranged diagonally. The segment positioning device 2 for the two cameras is located on the centerline of the crane. The positional relationship between the cameras and the center of the crane is calibrated. Because the cameras 1 and segment positioning device 2 are intersected, more segment feature information can be obtained. The spatial attitude of the segment is established using the binocular camera to correct the crane's attitude and prevent tilting. The left and right cameras respectively acquire segment images within their current field of view. The images are sent to a target recognition network to obtain the pixel coordinates of the segment holes. The recognized image coordinates are then converted to camera coordinates using a camera model to obtain a local point cloud of the segment. A point cloud evaluation function is used to select and calculate the three-dimensional points representing the positioning hole deviation features. The positional and attitude deviations between the crane positioning pin and the segment positioning hole are calculated in the same coordinate system, and the deviations are transmitted to the PLC central control system.

[0086] As a preferred embodiment of the automated tunnel segment hoisting system, it includes a tunnel segment placement and positioning system. This system calibrates the position of the tunnel segment transport vehicle relative to a camera. The tunnel segment is hoisted above the transport vehicle. Images captured by the camera identify the target placement position of the transport vehicle. A camera model calculates the deviation between the current crane and the transport vehicle, and this deviation is transmitted to the PLC central control system. The positioning markers on the transport vehicle can be replaced with any other target object different from the scene, such as marker lines, contour features, or indirect markers.

[0087] As a preferred embodiment of the automatic tunnel segment hoisting system, it includes a tunnel segment transportation system. After the tunnel segment is placed at the target position, it receives a control command from the PLC central control system and transports the tunnel segment to the set position.

[0088] The PLC central control system stores the position information of each crane lowering operation, receives and sends control commands in real time, and maintains the normal operation of the entire system. It monitors the operating status of each module system, determines whether it is in normal operating condition, and sends the system status to the control interface in real time.

[0089] As a preferred implementation of the automated tunnel segment hoisting system, such as Figure 2 and Figure 7 As shown, the two cameras 1 of the binocular camera are pinhole cameras. The pinhole cameras are mounted on the crane 7 by a protective device. The protective device includes a protective cover 3 set on the outer periphery of the pinhole camera. An annular light source 4 is set between the inner ring of the protective cover 3 and the outer ring of the pinhole camera. A gas flushing mechanism facing the pinhole camera and the annular light source is set on the protective cover 3.

[0090] Furthermore, the gas flushing mechanism is angle-adjustable relative to the pinhole camera and two mechanisms are provided, symmetrically arranged on both sides of the protective cover 3 about the pinhole camera. Each gas flushing mechanism includes a nozzle connected to the gas source device 5. The nozzles 6 of the two gas flushing mechanisms are coaxially mounted and at the same height. The nozzles 6 are tilted upwards, blowing airflow toward the pinhole camera and the ring light source 4. The protective cover 3 is preferably made of high-strength glass to prevent liquid from entering during operation and to protect the monitoring device from damage. The glass cover is installed at a height lower than the axis of the nozzles 6, at a certain tilt angle to guide the flushing gas. The high-speed gas flow ensures the clarity of the field of view and the brightness of the light source.

[0091] In a preferred embodiment of the automatic segment hoisting system, the hoisting rope of the hoist is equipped with an absolute encoder to monitor the rope length and determine whether the hoist is tilted; or, laser rangefinders are installed around the hoist to obtain the azimuth coordinates, calculate the current position of the hoist relative to the segment, and determine whether it is tilted.

[0092] All aspects not detailed in this invention are conventional technical means known to those skilled in the art.

[0093] The above content shows and describes the basic principles, main features, and beneficial effects of the present invention. The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatically hoisting tunnel segments, characterized in that... Includes the following steps: A. Establish a segment type database based on the geometric characteristics of different segment categories. Identify segment types using images captured by a binocular camera on the crane and a segment type identification network model. Establish a segment capture order database. B. Calculate the height of the tunnel segment in the camera coordinate system using the images acquired by the binocular camera, correct the current position of the crane based on the height value and the corresponding tunnel segment structural parameters, and record the three-dimensional coordinate value of the crane after correction. C. Based on the corrected crane position, the image location area of ​​the segment positioning hole is obtained by combining the segment image acquired by the binocular camera with the segment type identification network model. The coordinates of the image location area are transformed into three-dimensional coordinates in the camera coordinate system. The feature points in the image location area are evaluated by the feature evaluation function to obtain the positioning coordinates of the segment positioning hole. D. Calculate the deviation between the coordinate position of the positioning hole in the camera coordinate system and the coordinate position of the crane positioning pin in the camera coordinate system, correct the current attitude of the crane, and control the crane to complete the engagement of the crane positioning pin with the segment positioning hole. The feature evaluation function is: K1 and K2 are adjustable parameters that can be adjusted according to different scenarios to balance distance evaluation and data distribution evaluation; [X] i Y i For each feature point, [X] m Y m [a, b] represents the geometric center point of the identified image location region, [a, b] represents the image coordinates of the current feature point, g represents the image grayscale value of the current point, and n is the current feature point [X]. i Y i Relative to the geometric center point [X] m Y m The total number of feature points in the distance distribution, where N is the total number of feature points.

2. The automatic segment hoisting method according to claim 1, characterized in that: In step A, a segment type database is established based on the geometric characteristics of different types of segments. The binocular camera on the crane scans the uppermost segment in different segment stacks, and the collected segment images are input into the segment type identification network model to obtain the corresponding segment type. A segment grabbing order library is established based on the identified segment type and transmitted to the PLC central control system.

3. The automatic segment hoisting method according to claim 2, characterized in that: In step B, based on the segment grabbing sequence library, the PLC central control system controls the crane to move above the target segment. The binocular camera on the crane acquires images, calculates the parallax of the common area of ​​the left and right cameras, calculates the height value of the segment in the camera coordinate system, and corrects the current position of the crane according to the calculated height value and the corresponding segment structure parameters. If the current position of the crane exceeds the allowable tilt range, the PLC central control system corrects the position of the crane and controls the crane to be lowered to the corrected precise positioning height, and records the corrected three-dimensional coordinate value of the crane.

4. The automatic segment hoisting method according to any one of claims 1-3, characterized in that: In step C, based on the corrected crane position, the left and right cameras acquire images of the tunnel segment. The acquired images are input into the target recognition network to obtain the image location areas of the left and right positioning holes on the tunnel segment. Based on the image coordinates of the positioning holes and the distance between the binocular camera and the tunnel segment obtained from the PLC central control system, the coordinates of the image location areas are transformed into three-dimensional coordinates in the camera coordinate system. The feature points in the image location areas are evaluated through a feature evaluation function to obtain the positioning coordinates of the positioning holes.

5. The automatic segment hoisting method according to claim 4, characterized in that: In step D, the deviation between the coordinate position of the positioning hole in the camera coordinate system and the coordinate position of the crane positioning pin in the camera coordinate system is calculated and transmitted to the PLC central control system. The PLC central control system sends control commands to the crane to correct the current posture of the crane and control the crane to complete the engagement of the crane positioning pin with the segment positioning hole. After the crane reaches the lowering height, it grabs the segment and transports it to the predetermined position.

6. The automatic segment hoisting method according to claim 5, characterized in that: In steps C and D, the coordinates of the segment positioning hole in the camera coordinate system are calculated using the camera model. The initial center coordinates of the positioning hole are calculated based on the acquired feature point set. The initial center coordinates of the positioning hole and the feature point set are input into a feature evaluation function to evaluate the feature points and obtain the feature point with the maximum value. The precise coordinates of the positioning hole are then calculated based on the acquired feature point set. The coordinates of one camera are transformed to the coordinate system of the other camera using transformation matrices from the left and right cameras. The deviation includes the positional deviation of the identified segment positioning target relative to the target positioning [d]. x d y ] and rotational deviation [d r The calculation formula is as follows: dr = cos(V) 吊机 Position, V mark position), dx, dy = f(V crane position, V mark position).

7. The automatic segment hoisting method according to claim 6, characterized in that: The predetermined position is the placement position of the tunnel segment on the tunnel segment transport vehicle. After the crane grabs the tunnel segment and before it is lifted to the placement position, the positioning camera of the tunnel segment transport vehicle captures an image of the current tunnel segment transport vehicle, identifies the placement position mark on the tunnel segment transport vehicle, and calculates the plane coordinates P of the placement mark in the camera coordinate system. S [X, Y], transform the position coordinates of the crane and the segment transport vehicle to the same coordinate system, and calculate the position of the crane and the current rotational deviation of the segment transport vehicle [d]. r ] and horizontal deviation [d x d y The deviation is transmitted to the PLC central control system to adjust the crane's planar deviation. x d y ] and rotational deviation [d r After the crane is aligned, the PLC central control system controls the crane to lift the tunnel segment to the segment placement position, and then the crane returns to the ready-to-work position.

8. The automatic segment hoisting method according to any one of claims 1-3 and 5-7, characterized in that: The image location area is a rectangular area containing the target. The image location area is divided into three sub-regions according to the 120-degree arithmetic progression. The distance and height of the binocular camera from the tube segment are obtained from the PLC central control system. The image coordinates of the three sub-regions are converted into three-dimensional coordinates in the camera coordinate system. Each feature point is evaluated by the feature evaluation function D to obtain the set of positioning coordinates of the target positioning hole. The center coordinates are fitted based on the feature points of the three sub-regions, which are the coordinates of the target positioning hole.

9. The automatic segment hoisting method according to claim 8, characterized in that: The network model for identifying the type of pipe segment is designed as a target recognition network structure Model1, using a convolutional network as the feature extraction framework. When identifying the type of pipe segment, the features extracted by the network are used as input, and a softmax classifier is used to classify the features to obtain the type of the current pipe segment. There is no need to perform position regression on the pipe segment markers; only the target category needs to be identified.

10. The automatic segment hoisting method according to claim 9, characterized in that: During the segment grabbing and positioning and segment placement positioning, since it is necessary to identify the image coordinates of the segment hole and the segment transport vehicle placement position, Model 2 is constructed by adding the target image position deviation to the loss function based on Model 1. The extracted features are input into the target classifier and the position regressor to achieve target object classification and position regression.

11. The automatic segment hoisting method according to any one of claims 1-3, 5-7, and 9-10, characterized in that: The positioning holes can also be identified in the following ways: the point cloud of the current scene is obtained by the lidar on the mounting surface of the crane base, and the positioning holes are identified by the geometric features of the positioning holes; or the positioning holes are indirectly identified by placing markers around them and identifying the corresponding markers.

12. An automatic segment hoisting system, characterized in that: The system includes the crane, binocular camera, and segment transport vehicle positioning camera as described in claim 11. All three components are connected to a PLC central control system. The PLC central control system controls the binocular camera to acquire images and identifies the target segment type from the segment type database using a segment type identification network model. The system also controls the binocular camera to acquire images and obtains the positioning coordinates of the segment positioning holes using a feature evaluation function. The PLC central control system calculates the deviation between the positioning coordinates of the positioning holes and the coordinates of the crane positioning pins. Based on this deviation, the PLC central control system corrects the crane posture to complete the engagement between the crane positioning pins and the segment positioning holes.

13. The automatic segment hoisting system according to claim 12, characterized in that: The binocular camera is installed at the bottom of the crane, with the two cameras in the binocular camera being diagonally distributed, and the segment positioning device of the two cameras being located on the centerline of the crane.

14. The automatic segment hoisting system according to claim 12 or 13, characterized in that: The two cameras of the binocular camera are pinhole cameras. The pinhole cameras are mounted on a crane by a protective device. The protective device includes a protective cover set around the pinhole camera. An annular light source is set between the inner ring of the protective cover and the outer ring of the pinhole camera. The protective cover is provided with a gas flushing mechanism facing the pinhole camera and the annular light source.

15. The automatic segment hoisting system according to claim 14, characterized in that: The gas flushing mechanism is adjustable in angle relative to the pinhole camera and there are two of them. The two gas flushing mechanisms are symmetrically arranged on both sides of the protective cover about the pinhole camera.

16. The automatic segment hoisting system according to claim 15, characterized in that: Each gas flushing mechanism includes a nozzle connected to a gas source device. The nozzles of the two gas flushing mechanisms are mounted coaxially and at the same height. The nozzles are tilted upwards to blow airflow toward the pinhole camera and the ring light source.

17. The automatic segment hoisting system according to any one of claims 12-13 and 15-16, characterized in that: The crane's crane ropes are equipped with absolute encoders to monitor the rope length and determine whether the crane is tilted; or the crane is equipped with laser rangefinders around its perimeter to obtain azimuth coordinates, calculate the crane's current pose relative to the tunnel segment, and determine whether it is tilted.

Images