An intelligent detection method for water leakage in shield tunnel lining

Abstract

The present invention proposes an intelligent detection device and method for shield tunnel lining water leakage, which relates to the field of civil engineering operation and maintenance; it includes a lining water leakage multimodal information collection device, which includes a laser radar, a visible light camera and a handheld base, and the laser radar and the visible light camera are coaxially fixed on the handheld base through a pan / tilt, an adapter plate and bolts; the handheld base is also provided with a power supply, an edge computing device and a display device. The present invention adopts an established fast segmentation-feature quantization framework for water leakage images, and segments the water leakage in the visible light image based on the semantic segmentation method to obtain a water leakage target mask. The point cloud of the water leakage area is extracted based on the target mask, and the surface fitting algorithm is used to perform surface fitting on the water leakage area. The quantitative parameters such as the perimeter, area, and maximum axis length of the fitting surface are quickly and accurately calculated through the feature quantization algorithm.

Description

Technical Field

[0001] The present invention relates to the field of urban underground rail transit tunnel operation and maintenance, and in particular to an intelligent detection device and method for water leakage in a shield tunnel lining. Background Art

[0002] With the increase in computing power and miniaturization of scientific computing equipment, real-time identification of water seepage based on image information of tunnel lining is a feasible method to replace manual detection. Traditional image processing algorithms based on pixel values ​​and target detection methods based on deep neural networks have been tried to be applied to tunnel water seepage target detection. Traditional image processing algorithms classify lining image pixels by manually setting filters, pixel thresholds, and pixel grouping values, and select pixels that may be water seepage areas. The deep neural network method is based on existing image big data, learns the characteristics of water seepage targets, and judges the location of water seepage in the image based on prior knowledge when processing new images. However, the lighting conditions in the tunnel are complex, and the shooting parameters of different images vary greatly, which increases the difficulty of manually setting parameters in traditional algorithms. The deep neural network can automatically learn the characteristics of water seepage targets under different parameters. Compared with traditional methods, it has better generalization performance and is more suitable for target detection tasks in tunnel environments. However, the deep neural network can only realize target recognition at the pixel level, and cannot correspond the pixel size to the actual space size and calculate quantitative indicators.

[0003] Therefore, a shield tunnel lining water leakage intelligent detection device and detection method are provided to solve the above problems. Summary of the invention

[0004] The present invention provides an intelligent device and detection method for shield tunnel lining water leakage, designs a joint calibration scheme for tunnel usage scenarios, and outputs an accurate camera internal and external parameter matrix based on a high-precision calibration plate.

[0005] To achieve the above-mentioned object, the present invention provides an intelligent detection device for lining water leakage of a shield tunnel, comprising a multi-modal information collection device for lining water leakage, the multi-modal information collection device for lining water leakage comprising a laser radar, a visible light camera and a handheld base, the laser radar and the visible light camera are coaxially fixed on the handheld base through a pan / tilt, an adapter plate and bolts;

[0006] The handheld base is also equipped with a power supply, edge computing device and display device.

[0007] A detection method of an intelligent detection device for shield tunnel lining water leakage, specifically comprising the following steps:

[0008] S1: Establish a point cloud-visible light image joint calibration method. The laser radar and the visible light camera are calibrated with external parameters through the joint calibration method, so that the point cloud data taken by the laser radar and the visible light image taken by the visible light camera have a spatial correspondence relationship;

[0009] S2: After the calibration is completed, the external parameter calibration matrix of the laser radar and camera is obtained;

[0010] S3: Through the calibration matrix and interpolation method, the spatial position of each pixel in the visible light image is calculated based on the spatial three-dimensional coordinates of the point cloud;

[0011] S4: Use the image semantic segmentation method based on YOLOv8 to perform semantic segmentation on the visible light image and output the water leakage target mask;

[0012] S5: based on the water leakage target mask, extract the quantitative information of the point cloud data corresponding to the water leakage target;

[0013] S6: Using a two-dimensional surface fitting method, based on the extracted point cloud image quantization information corresponding to the water leakage target, the spatial shape of the water leakage target is fitted, the corresponding quantization parameters are calculated and output.

[0014] Preferably, in step S1, the laser radar adopts a radar coordinate system, and the coordinate axis passes through (x L ,y L , z L ) indicates that the coordinate origin is the physical imaging center of the laser radar;

[0015] The visible light camera uses a camera coordinate system, whose coordinate axes are (x C ,y C , z C ) indicates that the coordinate origin is the physical imaging center of the visible light camera;

[0016] Visible light images use an image coordinate system, where the coordinate axes are represented by (x, y) and the origin of the coordinates is the point where the optical axis intersects the image;

[0017] The pixels in the visible light image use pixel coordinates, and their coordinates are expressed as (u, v), and the origin of the coordinates is the upper left corner of the visible light image;

[0018] A checkerboard calibration plate is used to jointly calibrate the lidar and camera. The material of the calibration plate is set to aviation aluminum, the size is set to: 1 meter × 1 meter, and the flatness is ±0.03 mm; the size of the checkerboard in the checkerboard calibration plate is set to 10 cm × 10 cm, and the accuracy of the checkerboard is ±0.001 mm.

[0019] Preferably, the radar coordinate system and the camera coordinate system are calibrated by performing a linear transformation or a rotation calibration matrix plus a translation calibration matrix;

[0020] For coordinate system rotation, the original coordinate axes (x, y, z) are rotated by (j, a, q) to obtain the rotated coordinate axes (x', y', z'). The relationship between the rotated coordinates and the original coordinates is expressed as:

[0021]

[0022]

[0023]

[0024] The rotation matrices of the three coordinate axes are represented as R x , R y and R z , the overall rotation matrix R of the coordinate axis is expressed as: R = R x R y R z ;

[0025] The process of converting the radar coordinate system to the camera coordinate system is expressed as follows:

[0026]

[0027] After the form transformation, we get:

[0028]

[0029] Among them, R represents the rotation calibration matrix with a size of (3×3), and T represents the translation calibration matrix with a size of (3×1).

[0030] Preferably, during the visible light camera shooting process, the three-dimensional spatial coordinates are projected into two-dimensional plane coordinates; the origin of the camera coordinate system is O C , coordinate axis x C ,y C The coordinate axes x and y of the image coordinate system are parallel, and the coordinate axis z of the camera coordinate system is parallel. C Perpendicular to the xy plane and pointing outside the xy plane; the origin O of the image coordinate system falls on the coordinate axis z of the camera coordinate system C On, and the corresponding coordinates in the camera coordinate system are (0, 0, f), where f is the focal length of the camera;

[0031] Point P in the camera coordinate system C (x c ,y c , z c ) is projected into the image coordinate system, the projection point is P(x, y), and the following equation is obtained based on the perspective relationship:

[0032]

[0033] According to the above coordinate relationship, the corresponding coordinate axis relationship is:

[0034]

[0035] Convert to matrix form:

[0036]

[0037] After the above transformation, the spatial coordinates are transformed into the plane coordinates of the image coordinate system, and the transformed image coordinate system is transformed with the pixel coordinate system to obtain the relationship between the pixel point position and the actual length of the space;

[0038] In actual imaging, the pixel coordinate axes uv are at an angle η and are not perpendicular to each other. When converting between the image coordinate system and the pixel coordinate system, the conversion matrix is ​​based on plane translation and angle transformation, as follows:

[0039]

[0040]

[0041] Among them, u and v are the pixel coordinates after conversion, dx and dy are the lengths of the image coordinate system represented by each pixel, u0 and v0 are the coordinates after the origin O of the image coordinate system is projected to the pixel coordinate system;

[0042] Convert to matrix form:

[0043]

[0044] The radar coordinate system of the laser radar is converted to the pixel coordinate system of the image. The overall conversion relationship is expressed as:

[0045]

[0046] Among them, the two matrices in front of the radar coordinate system are called the camera's intrinsic parameter matrix and the camera's extrinsic parameter matrix respectively.

[0047] Preferably, in the calibration scene of step S2, the calibration distance is set to 4 meters, 7 meters or 10 meters;

[0048] At the calibration distance, adjust the position and angle of the calibration plate in the camera field of view, and place the calibration plate at the upper left, upper, upper right, left middle, middle, right middle, lower left, lower, and lower right positions of the visible light camera, respectively, take multiple groups of visible light images or point cloud data pairs, and calibrate the visible light images and point cloud data pairs through the calibration algorithm, and output the calibrated external parameter matrix;

[0049] Before calibrating the external parameter matrix, the lens distortion is corrected using the following distortion correction formula:

[0050] Radial distortion:

[0051] Where (x, y) is the original pixel coordinate; (x rev ,y rev ) is the pixel coordinate after image distortion restoration; r is the distance between the pixel coordinate and the geometric center of the image; k1, k2 and k3 are radial distortion coefficients, which are obtained during camera calibration;

[0052] Tangential distortion:

[0053] Where p1 and p2 are the tangential distortion coefficients; by solving, we can get r 2 =x 2 +y 2 ;

[0054] The calibration extrinsic parameter matrix is:

[0055]

[0056] Among them, z C is the z-axis of the camera coordinate system; (u, v) is the pixel position in the image; u0 and v0 are the coordinates of the image coordinate system origin O projected to the pixel coordinate system; f x and f y is the length of the focal length in the x and y axes described in pixels; γ is the coordinate axis tilt parameter; R is the rotation matrix of size (3×3), T is the translation matrix of size (3×1); x L ,y L , z L Represents the coordinate axes in the lidar coordinate system.

[0057] Preferably, in step S4, the following steps are specifically included:

[0058] S41: Collect visible light images of water seepage for multiple shield tunnel projects and build image datasets;

[0059] S42: using a data enhancement algorithm to enhance the training images in the image data set, wherein the enhancement method includes adding noise, shading transformation, random scaling and rotation;

[0060] S43: Enhanced water leakage image dataset, training YOLOv8 network, and outputting the network parameter file after training;

[0061] S44: Deploy in the C++ environment on the edge computing device based on the network parameter file;

[0062] S45: A semantic segmentation method is run on the edge computing device, where the semantic segmentation method includes a pre-processing module, a prediction network module, and a post-processing module.

[0063] Preferably, in step S43, a YOLO v8 semantic segmentation neural network training program is written in Python, and the training program includes: a data loading module, a model training module and a model post-processing module;

[0064] The data loading module randomly selects images from the dataset and scales them in a LetterBox manner, specifically: scaling the long side of the original image to the target size, and filling the part of the short side of the original image that is less than the target size with gray;

[0065] After the dataset image is input into the network, the model training module calculates the loss function value between the prediction result and the true value, and updates the neuron connection weights of the network by error back propagation until the prediction error of the network on the training dataset is less than the specified threshold;

[0066] In the model training module, the model output results are the predicted category and the predicted bounding box coordinates. The error of the predicted category is calculated using the BCE loss. The BCE loss value calculation formula is as follows:

[0067] l BCE =w n [y n ·logx n +(1-y n )·log(1-x n )]

[0068] Among them, w is the weight value, y is the true category of a certain type of target, and x is the predicted probability value of a certain type of target; the BCE loss value is the average value of multiple images in the training data set after network prediction;

[0069] The error of the predicted target bounding box is calculated by DFL loss and CIoU loss. The calculation formula of DFL loss is as follows:

[0070] DFL=-(y i+1 -y)log(S i )+(yy i log)log(S i+1 )

[0071] Among them, y is the coordinate of the corner point of the bounding box to be calculated, y i+1 and i is a fixed reference value;

[0072] The calculation formula of CIoU loss is as follows:

[0073]

[0074]

[0075]

[0076]

[0077] Among them, b and b gt are the center point coordinates of the predicted box and the real box, respectively, c is the diagonal distance between the predicted box and the smallest outer rectangular box of the real box, w, h, w gt 、h gt They are the length and width of the predicted box and the true box respectively;

[0078] The loss function used by the YOLO v8 program is the sum of the above two loss functions, expressed as:

[0079]

[0080] The post-model processing module quantifies the parameters in the model training process and outputs them in the form of data graphs. The parameters output by the module are: all And mIoU; where mIoU represents the comprehensive performance of the model on the entire training image dataset, and the calculation formula is:

[0081]

[0082] Among them, IoU n Represents the IoU value between the predicted target and the true target in each image in the dataset.

[0083] Preferably, in step S44, after the model training is completed, the Python format is converted into the .onnx mode through the Python Pytorch library, including the following steps:

[0084] S441: The best overall performance during training and the smallest l all And a set of model parameter values ​​with the largest mIoU are exported and stored as a Pytorch-based .pt format file;

[0085] S442: Use the torch.onnx.export function of the Pytorch library to save the .pt file as a .onnx format file;

[0086] S443: Convert the .onnx format file to an engine and deploy it to the edge computing device;

[0087] S444: In the edge computing device, reasoning is performed on the converted model based on the C++ environment, specifically including the following steps:

[0088] S4441: The edge computing device reads the video stream captured by the visible light camera and divides it into image frames;

[0089] S4442: The divided image frame is input into the inference model in the C++ environment, and the inference model outputs the prediction result of the image frame, and the prediction result includes the outline and type of the water leakage target in the image frame;

[0090] S4443: The edge computing device synchronously reads the point cloud data obtained by the laser radar scanning, and spatially projects the point cloud according to the optical axis direction of the visible light camera based on the joint calibration matrix;

[0091] S4444: Based on the plane pixel outline of the water leakage target, in the point cloud projection data, select the points within the water leakage outline for output. The output data is the position coordinates of the water leakage target in the real space, in the form of an array composed of several groups of three-dimensional space coordinates, expressed as:

[0092] Cloud=[(x1,y1,z1),(x1,y1,z1),...,(x n ,y n ,z n )]

[0093] Among them, x, y, and z are the three-dimensional coordinates of the midpoint in space. The two-dimensional interpolation method is used to fit these coordinate values, and the shape of the leakage target in the three-dimensional space is quantitatively obtained. The integral method is used to quantitatively obtain the area of ​​the leakage shape surface.

[0094] Preferably, in step S45, the pre-processing module pre-processes the visible light image, unifies the size and channel, and inputs it into the network; the prediction network module predicts and obtains the water leakage target information; the post-processing module outputs the quantitative parameters of the water leakage target based on the point cloud data;

[0095] The post-processing module superimposes the visible light image, water leakage target mask and water leakage quantitative parameters and outputs them to the display device to realize real-time detection of tunnel lining water leakage targets.

[0096] Therefore, the present invention adopts the above-mentioned intelligent detection method for shield tunnel lining leakage, which has the following beneficial effects:

[0097] (1) The present invention has developed a multimodal information acquisition device for lining water leakage, which collects point clouds and visible light images of the lining water leakage area of ​​the shield tunnel based on laser radar and visible light camera.

[0098] (2) The present invention proposes a point cloud-visible light image joint calibration method. Based on the above-mentioned information acquisition equipment, by taking images and point clouds of the calibration plate in multiple directions and angles, the distortion of the visible light camera is corrected, and the external parameters of the lidar and visible light camera are calibrated, so that the pixels in the visible light image can be spatially corresponded to the point cloud.

[0099] (3) The present invention establishes a fast segmentation-feature quantization framework for water leakage images. The water leakage in the visible light image is segmented based on the semantic segmentation method to obtain the water leakage target mask. Then, the point cloud of the water leakage area is extracted based on the target mask, and the surface fitting algorithm is used to perform surface fitting on the water leakage area. Finally, the quantitative parameters such as the perimeter, area, and maximum axis length of the fitting surface are quickly and accurately calculated through the feature quantization algorithm.

[0100] (4) The present invention superimposes and displays the visible light image, the water leakage target mask and the water leakage quantitative parameters, so that engineering personnel can check the shield tunnel lining water leakage detection results in real time.

[0101] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] Figure 1 It is a schematic diagram of a process of an intelligent detection method for water leakage in a shield tunnel lining according to the present invention;

[0103] Figure 2 It is a component diagram of the shield tunnel lining visible light image and point cloud data acquisition and processing equipment in the present invention;

[0104] Figure 3 It is a calibration board style diagram in the data joint calibration method provided in an embodiment of the present invention;

[0105] Figure 4 It is the training set image data enhancement process in the semantic segmentation method of the present invention;

[0106] Figure 5 This is the training process of the model in the Python environment in the semantic segmentation method of the present invention;

[0107] Figure 6 It is a flow chart of semantic segmentation and quantitative feature extraction reasoning of the present invention;

[0108] Figure 7 It is a schematic diagram of the multi-modal information collection device for lining water leakage of the present invention;

[0109] Figure Annotation

[0110] 1. Visible light camera; 2. LiDAR; 3. Adapter plate; 4. Handheld base; 5. Main handle; 51. Auxiliary handle; 6. Edge computing device; 7. Power supply; 8. Gimbal; 9. Visible light camera. DETAILED DESCRIPTION

[0111] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0112] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0113] Example

[0114] like Figure 1 As shown, the present invention provides a shield tunnel lining water leakage intelligent detection device, including lining water leakage multi-modal information collection equipment such as Figure 2 As shown, the lining water leakage multimodal information collection equipment includes a laser radar 2, a visible light camera 9 and a handheld base. The laser radar 2 and the visible light camera 9 are coaxially fixed on the handheld base 4 through a pan-tilt 8, an adapter plate 3 and bolts; the visible light camera 9 and the laser radar 2 are respectively fixed on two pan-tilts 8 through 1 / 4-20UNC bolts, and by adjusting the horizontal and vertical directions of the pan-tilt 8, the optical axes of the visible light camera 9 and the laser radar 2 are made roughly parallel and point to the front of the handheld base 4.

[0115] The handheld base 4 is made of aluminum profiles and engineering plastics, and includes a main handle 5, an auxiliary handle 51, a device stand, a display stand and a battery compartment. The main handle 5 and the auxiliary handle 51 are used to allow the user to hold them conveniently. The device stand is used to firmly install the visible light camera 1, the edge computing device 6 and the power supply 7 so that they maintain a stable working state during operation. The display stand is used to fix the display and facilitate adjustment of the viewing angle of the display. The battery compartment is used to store and protect the power supply 7.

[0116] The handheld base 4 is also provided with a power supply 7, an edge computing device 6 and a display device.

[0117] The power supply 7 is powered by 6 18650 batteries in 3 groups connected in series, and outputs 12V DC through the power protection board. The output interface of the power protection board is an XT60 interface, and an XT60 to DC 5.5×2.5 power cable is used to connect the power supply to the power-consuming device. The XT60 interface can be easily connected and disconnected. In actual use, the backup power supply can be quickly replaced for the device to improve work efficiency.

[0118] The edge computing device 6 is installed on the handheld base 4 through the bolt holes reserved on the shell, collects the visible light images and point cloud information collected by the visible light camera 1 and the laser radar 2, detects the shield tunnel lining water leakage targets, and calculates their quantitative information. The laser radar 2 is connected to the edge computing device 6 through a network cable, and the visible light camera 9 is connected to the edge computing device 6 through the CSI / MIPI interface. The edge computing device 6 is equipped with an external cooling fan to ensure its normal operation in a complex environment.

[0119] The display device includes a display, which is fixed to a display stand by bolts and connected to the edge computing device 6 by an HDMI cable, and the display stand is fixed to the handheld base 4 by bolts. By adjusting the direction and angle of the display stand, the user can clearly see the shield tunnel lining water leakage detection result processed by the edge computing device 6.

[0120] A detection method of an intelligent detection device for shield tunnel lining water leakage, specifically comprising the following steps:

[0121] S1: Establish a point cloud-visible light image joint calibration method. The laser radar and the visible light camera are calibrated with external parameters through the joint calibration method, so that the point cloud data taken by the laser radar and the visible light image taken by the visible light camera have a spatial correspondence. In step S1, the laser radar adopts the radar coordinate system, and the coordinate axis is (x L ,y L , z L ) indicates that the coordinate origin is the physical imaging center of the laser radar; that is, the light source, x L Axis horizontal forward, y L The axis is horizontal to the left, z L The axis is vertically upward, which conforms to the right-hand rule;

[0122] The visible light camera uses a camera coordinate system, whose coordinate axes are (x C ,y C , z C ) indicates that the coordinate origin is the physical imaging center of the visible light camera; that is, the optical center. C The y axis is parallel to the horizontal direction of the image. CThe axis is parallel to the vertical direction of the image, z C The axis is parallel to the optical axis of the camera and is C Axis, y C Axis vertical.

[0123] Visible light images use an image coordinate system, where the coordinate axes are represented by (x, y), and the origin of the coordinate system is the point where the optical axis intersects the image; that is, the camera principal point, which is generally the center of the image; the x and y of the image coordinate system both represent specific length units in the image;

[0124] The pixels in the visible light image use pixel coordinates, and their coordinates are expressed as (u, v), and the origin of the coordinates is the upper left corner of the visible light image;

[0125] Among all the above coordinate systems, only the radar coordinate system can truly reflect the real position of an object in three-dimensional space without any transformation. Therefore, by fixing the lidar and visible light camera and taking multiple sets of point cloud-visible light image data pairs, the transformation relationship matrix between the radar coordinate system, camera coordinate system, image coordinate system and pixel coordinate system can be solved, and the positions in different coordinate systems can be transformed through this matrix to achieve the ability to obtain spatial position based on pixel position.

[0126] The LiDAR and camera are jointly calibrated using a checkerboard calibration plate. Figure 3 As shown, the material of the calibration plate in the checkerboard calibration plate is set to aviation aluminum, the size is set to: 1 meter × 1 meter, and the flatness is ±0.03 mm; the size of the checkerboard in the checkerboard calibration plate is set to 10 cm × 10 cm, and the accuracy of the checkerboard is ±0.001 mm.

[0127] Calibration is achieved by linear transformation of the radar coordinate system and the camera coordinate system or by rotating the calibration matrix and translating the calibration matrix. First, calibrate the radar coordinate system and the camera coordinate system. Since the camera coordinate system is also based on the three-dimensional coordinates of the real space, calibration can be achieved by linear transformation of the two coordinate systems. In three-dimensional space, two similar coordinate systems can be transformed by combining rotation and translation. Therefore, in the process of calibrating the radar coordinate system and the camera coordinate system, it can also be done by rotating the calibration matrix and translating the calibration matrix.

[0128] For coordinate system rotation, the original coordinate axes (x, y, z) are rotated by (j, a, q) to obtain the rotated coordinate axes (x', y', z'). The relationship between the rotated coordinates and the original coordinates is expressed as:

[0129]

[0130]

[0131]

[0132] The rotation matrices of the three coordinate axes are represented as R x , R y and R z , the overall rotation matrix R of the coordinate axis is expressed as: R = R x R y R z ;

[0133] The process of converting the radar coordinate system to the camera coordinate system is expressed as follows:

[0134]

[0135] After the form transformation, we get:

[0136]

[0137] Among them, R represents the rotation calibration matrix with a size of (3×3), and T represents the translation calibration matrix with a size of (3×1).

[0138] During the shooting process of a visible light camera, external light enters the camera through a series of refractions of the lens and forms a planar image on the imaging element. This process can be described as the projection of three-dimensional spatial coordinates into two-dimensional plane coordinates; the origin of the camera coordinate system is O C , coordinate axis x C ,y C The coordinate axes x and y of the image coordinate system are parallel, and the coordinate axis z of the camera coordinate system is parallel. C Perpendicular to the xy plane and pointing outside the xy plane; the origin O of the image coordinate system falls on the coordinate axis z of the camera coordinate system C On, and the corresponding coordinates in the camera coordinate system are (0, 0, f), where f is the focal length of the camera;

[0139] Point P in the camera coordinate system C (x c ,y c , z c ) is projected into the image coordinate system, the projection point is P(x, y), and the following equation is obtained based on the perspective relationship:

[0140]

[0141] According to the above coordinate relationship, the corresponding coordinate axis relationship is:

[0142]

[0143] Convert to matrix form:

[0144]

[0145] After the above transformation, the spatial coordinates are converted into the plane coordinates of the image coordinate system. However, since the image coordinate system is defined as the spatial plane at the focal length of the camera, the unit of the converted coordinates is still the length unit millimeter. Therefore, it is also necessary to convert the converted image coordinate system with the pixel coordinate system to obtain the relationship between the pixel point position and the actual length of the space.

[0146] Due to camera manufacturing errors and other reasons, in actual imaging, the pixel coordinate axes uv are at an angle η and are not perpendicular to each other. Therefore, when converting between the image coordinate system and the pixel coordinate system, the conversion matrix is ​​based on plane translation and angle transformation, as follows:

[0147]

[0148]

[0149] Among them, u and v are the pixel coordinates after conversion, dx and dy are the lengths of the image coordinate system represented by each pixel, u0 and v0 are the coordinates after the origin O of the image coordinate system is projected to the pixel coordinate system;

[0150] Convert to matrix form:

[0151]

[0152] The radar coordinate system of the laser radar is converted to the pixel coordinate system of the image. The overall conversion relationship is expressed as:

[0153]

[0154] Among them, the two matrices in front of the radar coordinate system are called the camera's intrinsic parameter matrix and the camera's extrinsic parameter matrix. At this point, the image-point cloud coordinates under ideal conditions have been calibrated. In actual shooting, due to the limitations of lens materials, there is a certain difference between the virtual image formed by the camera in the image coordinate system and the real projection of the three-dimensional space object, which is lens distortion.

[0155] S2: After the calibration is completed, the external parameter calibration matrix of the laser radar and camera is obtained; in the calibration scene of step S2, a suitable scene is selected for calibration according to the tunnel working environment. Since the radius of an urban shield tunnel is generally 3 meters to 6 meters, when the staff conducts inspection in the tunnel, the possible location of the water leakage target is 3 meters to 10 meters. Therefore, when performing calibration, the calibration distance is set to 4 meters, 7 meters or 10 meters;

[0156] At the calibration distance, adjust the position and angle of the calibration plate in the camera field of view, and place the calibration plate at the upper left, upper, upper right, left middle, middle, right middle, lower left, lower, and lower right positions of the visible light camera, respectively, take multiple groups of visible light images or point cloud data pairs, and calibrate the visible light images and point cloud data pairs through the calibration algorithm, and output the calibrated external parameter matrix;

[0157] Before calibrating the external parameter matrix, the lens distortion is corrected using the following distortion correction formula:

[0158] Radial distortion:

[0159] Where (x, y) is the original pixel coordinate; (x rev ,y rev ) is the pixel coordinate after image distortion restoration; r is the distance between the pixel coordinate and the geometric center of the image; k1, k2 and k3 are radial distortion coefficients, which are obtained during camera calibration;

[0160] Tangential distortion:

[0161] Where p1 and p2 are the tangential distortion coefficients; by solving, we can get r 2 =x 2 +y 2 ; Through the above coordinate system conversion and distortion solution process, based on the point cloud-image data pairs taken by the lidar and visible light camera, the calibration of the camera's internal and external parameters can be completed, and the camera's internal and external parameter matrix and distortion parameters can be obtained.

[0162] The calibration extrinsic parameter matrix is:

[0163]

[0164] Among them, z C is the z-axis of the camera coordinate system; (u, v) is the pixel position in the image; u0 and v0 are the coordinates of the image coordinate system origin O projected to the pixel coordinate system; f x and f y is the length of the focal length in the x and y axes described in pixels; γ is the axis tilt parameter; R is the rotation matrix of size (3×3), T is the translation matrix of size (3×1); x L ,y L , z L Represents the coordinate axes in the lidar coordinate system.

[0165] S3: Through the calibration matrix and interpolation method, the spatial position of each pixel in the visible light image is calculated based on the spatial three-dimensional coordinates of the point cloud;

[0166] S4: Using the image semantic segmentation method based on YOLOv8, semantic segmentation is performed on the visible light image and a water leakage target mask is output; in step S4, the following steps are specifically included:

[0167] S41: collect visible light images of water leakage from multiple shield tunnel projects and construct an image data set; and use a manual marking method to mark the water leakage targets in the images in the data set, and the marking content is the type and outer contour of the water leakage area;

[0168] S42: using a data enhancement algorithm to enhance the training images in the image data set, wherein the enhancement method includes adding noise, shading transformation, random scaling and rotation;

[0169] S43: The enhanced water leakage image dataset is used to train the YOLOv8 network, and the network parameter file after training is output; in step S43, the YOLO v8 semantic segmentation neural network training program is written in Python language, and the training program includes: a data loading module, a model training module and a model post-processing module;

[0170] The data loading module randomly selects images from the dataset and scales them in a LetterBox manner, specifically: scaling the long side of the original image to the target size, and filling the part of the short side of the original image that is less than the target size with gray;

[0171] After the dataset image is input into the network, the model training module calculates the loss function value between the prediction result and the true value, and updates the neuron connection weights of the network by error back propagation until the prediction error of the network on the training dataset is less than the specified threshold;

[0172] In the model training module, the model output results are the predicted category and the predicted bounding box coordinates. The error of the predicted category is calculated using the BCE loss. The BCE loss value calculation formula is as follows:

[0173] l BCE =w n [y n ·logx n +(1-y n )·log(1-x n )]

[0174] Among them, w is the weight value, y is the true category of a certain type of target, and x is the predicted probability value of a certain type of target; the BCE loss value is the average value of multiple images in the training data set after network prediction;

[0175] The error of the predicted target bounding box is calculated using the DFL loss and CIoU loss. The calculation formula of the DFL loss is as follows:

[0176] DFL=-(yi+1 -y)log(S i )+(yy i log)log(S i+1 )

[0177] Among them, y is the coordinate of the corner point of the bounding box to be calculated, y i+1 and i is a fixed reference value;

[0178] The calculation formula of CIoU loss is as follows:

[0179]

[0180]

[0181]

[0182]

[0183] Among them, b and b gt are the center point coordinates of the predicted box and the real box, respectively, c is the diagonal distance between the predicted box and the smallest outer rectangular box of the real box, w, h, w gt 、h gt They are the length and width of the predicted box and the true box respectively;

[0184] The loss function used by the YOLO v8 program is the sum of the above two loss functions, expressed as:

[0185]

[0186] The function of the post-model processing module is to quantify the parameters in the model training process and output them in the form of data graphs, so that users can view the real-time training status of the model. The parameters output by the module are: all And mIoU. Among them, mIoU represents the comprehensive performance of the model on the entire training image dataset, and its calculation formula is:

[0187]

[0188] Among them, IoU n Represents the IoU value between the predicted target and the true target in each image in the dataset.

[0189] S44: Based on the network parameter file, deploy in the C++ environment on the edge computing device; in step S44, after the model training is completed, convert the Python format to .onnx mode through the Python Pytorch library, including the following steps:

[0190] S441: The best overall performance during training and the smallest l all And a set of model parameter values ​​with the largest mIoU are exported and stored as a Pytorch-based .pt format file;

[0191] S442: Use the torch.onnx.export function of the Pytorch library to save the .pt file as a .onnx format file;

[0192] S443: Convert the .onnx format file to an engine and deploy it to the edge computing device;

[0193] S444: In the edge computing device, reasoning is performed on the converted model based on the C++ environment, specifically including the following steps:

[0194] S4441: The edge computing device reads the video stream captured by the visible light camera and divides it into image frames;

[0195] S4442: The divided image frame is input into the inference model in the C++ environment, and the inference model outputs the prediction result of the image frame, which includes the outline and type of the water leakage target in the image frame;

[0196] S4443: The edge computing device synchronously reads the point cloud data obtained by the laser radar scanning, and spatially projects the point cloud according to the optical axis direction of the visible light camera based on the joint calibration matrix;

[0197] S4444: Based on the plane pixel outline of the water leakage target, in the point cloud projection data, select the points within the water leakage outline for output. The output data is the position coordinates of the water leakage target in the real space, in the form of an array composed of several groups of three-dimensional space coordinates, expressed as:

[0198] Cloud=[(x1,y1,z1),(x1,y1,z1),...,(x n ,y n ,z n )]

[0199] Among them, x, y, and z are the three-dimensional coordinates of the midpoint in space. The two-dimensional interpolation method is used to fit these coordinate values, and the shape of the leakage target in the three-dimensional space is quantitatively obtained. The integral method is used to quantitatively obtain the area of ​​the leakage shape surface.

[0200] S45: The semantic segmentation method is run on the edge computing device. The semantic segmentation method includes a pre-processing module, a prediction network module and a post-processing module. In step S45, the pre-processing module pre-processes the visible light image, unifies the size and channel, and inputs it into the network; the prediction network module predicts the water leakage target information; the post-processing module outputs the quantitative parameters of the water leakage target based on the point cloud data;

[0201] The post-processing module superimposes the visible light image, water leakage target mask and water leakage quantitative parameters and outputs them to the display device to realize real-time detection of tunnel lining water leakage targets.

[0202] S5: based on the water leakage target mask, extract the quantitative information of the point cloud data corresponding to the water leakage target;

[0203] S6: Using a two-dimensional surface fitting method, based on the extracted point cloud image quantization information corresponding to the water leakage target, the spatial shape of the water leakage target is fitted, the corresponding quantization parameters are calculated and output.

[0204] Embodiment 1

[0205] Before using the lining leakage multimodal information collection equipment for real-time leakage detection, it is necessary to calibrate the visible light camera and lidar so that the pixel position in the image corresponds to the spatial point cloud coordinates, which is convenient for subsequent calculation of the quantitative characteristics of the leakage. Figure 3 The equipment is calibrated using the aluminum alloy black and white checkerboard calibration plate shown.

[0206] During calibration, one operator operates the lining water leakage multimodal information collection device, places the device on a stable plane, turns on the camera and lidar, and runs the calibration program. Another operator holds the calibration plate and moves it 4 meters, 7 meters, and 10 meters away from the device in the calibration area, and cooperates with the equipment operator to randomly move the calibration plate so that it is located at the upper left, upper, upper right, left middle, middle, right middle, lower left, lower, and lower right positions of the visible light camera's field of view. After the calibration plate operator moves to different positions, the calibration program is used to solve the camera's internal and external parameters. After the solution is completed, the calibration matrix is ​​input into the model inference program, and the corresponding three-dimensional point cloud can be obtained based on the water leakage mask in the image.

[0207] While the equipment is calibrated, the image semantic segmentation model is trained and deployed. First, the Labelme annotation software in the water leakage annotation module is used to annotate the water leakage area mask in the training set image. In the software, multiple straight lines are used to gradually surround the water leakage area to accurately obtain the water leakage contour. The annotated water leakage mask information is stored in a .JSON file. Then, the data conversion function in the image annotation module extracts the water leakage contour polylines in the .JSON file, and generates a .png image with corresponding type pixels as the image label. If there are n types of water leakage in the original image, the final .png image label has a total of n+1 pixels, namely pixels corresponding to type 1, pixels corresponding to type 2, ... pixels corresponding to type n, and pixels corresponding to the background.

[0208] The database construction module reads all the image files and annotation files generated above and outputs two data sets: a training set for network training and a validation set for network validation.

[0209] The role of the training set is to train the semantic segmentation network so that the network can learn the features in existing images and accurately describe the categories and contours when processing new images.

[0210] The role of the validation set is to evaluate the network training performance in real time during the training process and to determine the training status of the network.

[0211] After the data set is divided, the training set needs to be enhanced to increase the diversity of the training set data and enhance the generalization ability of the network during the training process. Figure 4 As shown. The training set images in the image database and their corresponding annotation information will be read by the image enhancement software. A random selection algorithm is used to select some image files for pixel-level transformation, such as adding noise, light and dark transformation, etc. For image noise, three noise generation methods, additive noise, multiplicative noise and quantization noise, are used to process the image to simulate image noise caused by various reasons. For each noise generation method, Gaussian noise, impulse noise (salt and pepper noise), Rayleigh noise, gamma noise, exponential noise and uniform noise are used for random generation. Among them, Gaussian noise is generated based on Gaussian distribution, and its distribution formula is as follows:

[0212]

[0213] After the noise is generated, it is normalized to the specified interval and added to the image by adding or multiplying the corresponding pixel values ​​of the original image, and finally output. For the image brightness change, the image is processed by brightness adjustment and contrast adjustment. In the brightness adjustment method, the global brightness parameter transformation method is adopted, and the implementation methods include linear transformation, logarithmic transformation and gamma correction. Among them, gamma correction is defined by the power law formula, which is expressed as follows:

[0214]

[0215] Among them, V out and V in They represent the output and input images respectively, A is the coefficient, which is usually 1, and γ is the coefficient, which is set according to the degree of adjustment of the image as needed.

[0216] Contrast adjustment is achieved by adjusting the image pixel values ​​within a single brightness range and the difference between different brightness ranges. The implementation methods include histogram equalization, adaptive histogram equalization, and contrast stretching. Through the above pixel processing methods, tunnel water leakage images obtained by various shooting devices and under various shooting conditions can be simulated.

[0217] In addition, the image enhancement software will further transform the image at the image level, such as random scaling, rotation, etc. The bilinear interpolation method is used to operate the image pixels to achieve scaling. The specific calculation method is as follows:

[0218]

[0219] Among them, x, y are the positions of the pixels to be found, Q ii is the adjacent pixel, x i ,y i Q ii The coordinate position of .

[0220] For the randomly reduced image, it will be placed in a canvas of the same size as the original image (the length and width of the matrix are the same), and the area left after the reduction will be filled with gray. For the enlarged image, the center of its image matrix is ​​aligned with the center of the matrix of the original image, and an image matrix of the same size as the original image is cut out on the new image. The purpose of using the image scaling method is to generate target images of various scales, so as to train the network's multi-scale target recognition ability. After the image is rotated, the resolution of the image is kept unchanged, and the blank pixels generated by the image rotation are filled with gray in the horizontal and vertical directions. The purpose of using image rotation transformation is to generate multi-directional targets and improve the network's detection ability for multi-directional targets. For the transformed image, the algorithm reads the corresponding water leakage information record file, and updates the position information of the minimum circumscribed rectangle of the water leakage in the record file according to the image transformation method, and outputs the modified information record file. Among them, for the randomly scaled image, the pixel position value in the information record file is replaced according to the position of the current target in the image. For the rotated image, the software will rotate the target detection frame in the information record file accordingly and map the coordinates to the new image. Then, the original target frame will be adjusted and output.

[0221] The enhanced water leakage image is input into the network training module, and the image segmentation model is trained, such as Figure 5 As shown. After the image is input into the network, the water leakage features it contains are extracted through the backbone network. The feature pyramid network fuses feature maps of different sizes to further enhance the water leakage information contained in the feature map. The network prediction head reads the enhanced feature map and predicts the category information of the water leakage and its segmentation information in the image. There will be differences between the actual category and spatial position of the water leakage target and the results predicted by the network prediction head. The loss function value is calculated based on these differences, and the loss function value is back-propagated in the network, so that the backbone network, feature pyramid, and network prediction head obtain error information and adjust their own weights accordingly, so that the subsequent prediction accuracy is increased. By setting a threshold for the above differences, the network training can be stopped when the network reaches an acceptable accuracy.

[0222] After the above network training is completed in the Python environment, the model parameter value with the smallest difference is obtained and saved as a .pt format file. Deploying this .pt file to the edge computing device through a series of processes can accelerate the model's reasoning process and achieve real-time detection. Use the model output function in the Torch library to convert the .pt format file to the .onnx general neural network model format. After that, convert the .onnx format file to an engine file. The engine file can be deployed in the C++ environment of the edge computing device. During reasoning, the operator can hold the above device and walk in the tunnel section facing the lining.

[0223] Therefore, the present invention adopts the above-mentioned intelligent detection device and method for water leakage in shield tunnel lining. The visible light camera and laser radar will respectively collect video data stream and point cloud data stream, and output them to the edge computing device. In the edge computing device, the video data stream will be decomposed into video frame images; the model deployed in the C++ environment reads the video frame image and outputs the semantic segmentation result of the image. Based on the semantic segmentation result of the image, the model intercepts the point cloud of the water leakage area on the point cloud at the corresponding time of the video frame. The model will perform surface fitting on the point cloud of the water leakage area and solve its quantitative information. The model will perform data superposition on the semantic segmentation result and quantitative information of the image, recombining the image into a video, and output it to the display, so that the operator can watch the water leakage detection results in real time.

[0224] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. An intelligent detection method for water leakage in shield tunnel lining, characterized by: It includes a lining water leakage multi-modal information collection device, which includes a laser radar, a visible light camera and a handheld base, and the laser radar and the visible light camera are coaxially fixed on the handheld base through a pan / tilt, an adapter plate and bolts; The handheld base is also equipped with a power supply, edge computing equipment and display equipment; The specific steps include: S1: Establish a point cloud-visible light image joint calibration method. The laser radar and the visible light camera are calibrated with external parameters through the joint calibration method, so that the point cloud data taken by the laser radar and the visible light image taken by the visible light camera have a spatial correspondence relationship; Calibrate the radar coordinate system and the camera coordinate system by linear transformation or rotation calibration matrix plus translation calibration matrix; For coordinate system rotation, the original coordinate axes (x, y, z) are rotated by (j, a, q) to obtain the rotated coordinate axes (x', y', z'). The relationship between the rotated coordinates and the original coordinates is expressed as: The rotation matrices of the three coordinate axes are represented as R x , R y and R z , the overall rotation matrix R of the coordinate axis is expressed as: R = R x R y R z ; The process of converting the radar coordinate system to the camera coordinate system is expressed as follows: After the form transformation, we get: Where R represents the rotation calibration matrix with a size of (3×3), and T represents the translation calibration matrix with a size of (3×1); S2: After the calibration is completed, the external parameter calibration matrix of the laser radar and camera is obtained; S3: Through the calibration matrix and interpolation method, the spatial position of each pixel in the visible light image is calculated based on the spatial three-dimensional coordinates of the point cloud; S4: Use the image semantic segmentation method based on YOLOv8 to perform semantic segmentation on the visible light image and output the water leakage target mask; S5: based on the water leakage target mask, extract the quantitative information of the point cloud data corresponding to the water leakage target; S6: Using a two-dimensional surface fitting method, based on the extracted point cloud image quantization information corresponding to the water leakage target, the spatial shape of the water leakage target is fitted, the corresponding quantization parameters are calculated, and output.

2. The intelligent detection method for shield tunnel lining water leakage according to claim 1 is characterized by: In step S1, the laser radar adopts the radar coordinate system, and the coordinate axis is (x L ,y L , z L ) indicates that the coordinate origin is the physical imaging center of the laser radar; The visible light camera uses a camera coordinate system, whose coordinate axes are (x C ,y C , z C ) indicates that the coordinate origin is the physical imaging center of the visible light camera; Visible light images use an image coordinate system, where the coordinate axes are represented by (x, y) and the origin of the coordinates is the point where the optical axis intersects the image; The pixels in the visible light image use pixel coordinates, and their coordinates are expressed as (u, v), and the origin of the coordinates is the upper left corner of the visible light image; A checkerboard calibration plate is used to jointly calibrate the lidar and camera. The material of the calibration plate is set to aviation aluminum, the size is set to: 1 meter × 1 meter, and the flatness is ±0.03 mm; the size of the checkerboard in the checkerboard calibration plate is set to 10 cm × 10 cm, and the accuracy of the checkerboard is ±0.001 mm.

3. The intelligent detection method for shield tunnel lining water leakage according to claim 2 is characterized by: During the shooting process of the visible light camera, the three-dimensional spatial coordinates are projected into the two-dimensional plane coordinates; the origin of the camera coordinate system is O C , coordinate axis x C ,y C The coordinate axes x and y of the image coordinate system are parallel, and the coordinate axis z of the camera coordinate system is parallel. C perpendicular to the xy plane and pointing out of the xy plane; The origin O of the image coordinate system falls on the coordinate axis z of the camera coordinate system C On, and the corresponding coordinates in the camera coordinate system are (0, 0, f), where f is the focal length of the camera; Point P in the camera coordinate system C (x c ,y c , z c ) is projected into the image coordinate system, the projection point is P(x, y), and the following equation is obtained based on the perspective relationship: According to the above coordinate relationship, the corresponding coordinate axis relationship is: Convert to matrix form: After the above transformation, the spatial coordinates are transformed into the plane coordinates of the image coordinate system, and the transformed image coordinate system is transformed with the pixel coordinate system to obtain the relationship between the pixel point position and the actual length of the space; In actual imaging, the pixel coordinate axes uv are at an angle η and are not perpendicular to each other. When converting between the image coordinate system and the pixel coordinate system, the conversion matrix is ​​based on plane translation and angle transformation, as follows: Among them, u and v are the pixel coordinates after conversion, dx and dy are the lengths of the image coordinate system represented by each pixel, u0 and v0 are the coordinates after the origin O of the image coordinate system is projected to the pixel coordinate system; Convert to matrix form: The radar coordinate system of the laser radar is converted to the pixel coordinate system of the image. The overall conversion relationship is expressed as: Among them, the two matrices in front of the radar coordinate system are called the camera's intrinsic parameter matrix and the camera's extrinsic parameter matrix respectively.

4. The intelligent detection method for shield tunnel lining water leakage according to claim 3 is characterized by: In step S2, in the calibration scene, the calibration distance is set to 4 meters, 7 meters or 10 meters; At the calibration distance, adjust the position and angle of the calibration plate in the camera field of view, and place the calibration plate at the upper left, upper, upper right, left middle, middle, right middle, lower left, lower, and lower right positions of the visible light camera, respectively, take multiple groups of visible light images or point cloud data pairs, and calibrate the visible light images and point cloud data pairs through the calibration algorithm, and output the calibrated external parameter matrix; Before calibrating the external parameter matrix, the lens distortion is corrected using the following distortion correction formula: Radial distortion: Where (x, y) is the original pixel coordinate; (x rev ,y rev ) is the pixel coordinate after image distortion restoration; r is the distance between the pixel coordinate and the geometric center of the image; k1, k2 and k3 are radial distortion coefficients, which are obtained during camera calibration; Tangential distortion: Where p1 and p2 are the tangential distortion coefficients; by solving, we can get r 2 =x 2 +y 2 ; The calibration extrinsic parameter matrix is: Among them, z C is the z-axis of the camera coordinate system; (u, v) is the pixel position in the image; u0 and v0 are the coordinates of the image coordinate system origin O projected to the pixel coordinate system; f x and f y is the length of the focal length in the x and y axes described in pixels; γ is the axis tilt parameter; R is the rotation matrix of size (3×3), T is the translation matrix of size (3×1); x L ,y L , z L Represents the coordinate axes in the lidar coordinate system.

5. The intelligent detection method for shield tunnel lining water leakage according to claim 1 is characterized by: In step S4, the following steps are specifically included: S41: Collect visible light images of water seepage for multiple shield tunnel projects and build image datasets; S42: using a data enhancement algorithm to enhance the training images in the image data set, wherein the enhancement method includes adding noise, shading transformation, random scaling and rotation; S43: Enhanced water leakage image dataset, training YOLOv8 network, and outputting the network parameter file after training; S44: Deploy in the C++ environment on the edge computing device based on the network parameter file; S45: A semantic segmentation method is run on the edge computing device, where the semantic segmentation method includes a pre-processing module, a prediction network module, and a post-processing module.

6. The intelligent detection method for shield tunnel lining water leakage according to claim 5 is characterized by: In step S43, a YOLO v8 semantic segmentation neural network training program is written in Python, and the training program includes: a data loading module, a model training module and a model post-processing module; The data loading module randomly selects images from the dataset and scales them in a LetterBox manner, specifically: scaling the long side of the original image to the target size, and filling the part of the short side of the original image that is less than the target size with gray; After the dataset image is input into the network, the model training module calculates the loss function value between the prediction result and the true value, and updates the neuron connection weights of the network by error back propagation until the prediction error of the network on the training dataset is less than the specified threshold; In the model training module, the model output results are the predicted category and the predicted bounding box coordinates. The error of the predicted category is calculated using the BCE loss. The BCE loss value calculation formula is as follows: l BCE =w n [y n ·logx n +(1-y n )·log(1-x n )]; Among them, w is the weight value, y is the true category of a certain type of target, and x is the predicted probability value of a certain type of target; the BCE loss value is the average value of multiple images in the training data set after network prediction; The error of the predicted target bounding box is calculated using the DFL loss and CIoU loss. The calculation formula of the DFL loss is as follows: DFL=-(y i+1 -y)log(S i )+(y-y i log)log(S i+1 ); Among them, y is the coordinate of the corner point of the bounding box to be calculated, y i+1 and i is a fixed reference value; The calculation formula of CIoU loss is as follows: Among them, b and b gt are the center point coordinates of the predicted box and the real box, respectively, c is the diagonal distance between the predicted box and the smallest outer rectangular box of the real box, w, h, w gt 、h gt They are the length and width of the predicted box and the true box respectively; The loss function used by the YOLO v8 program is the sum of the above two loss functions, expressed as: The post-model processing module quantifies the parameters in the model training process and outputs them in the form of data graphs. The parameters output by the module are: all And mIoU; where mIoU represents the comprehensive performance of the model on the entire training image dataset, and the calculation formula is: Among them, IoU n Represents the IoU value between the predicted target and the true target in each image in the dataset.

7. The intelligent detection method for shield tunnel lining water leakage according to claim 6 is characterized by: In step S44, after the model training is completed, the Python format is converted to .onnx mode through the Python Pytorch library, including the following steps: S441: The best overall performance during training and the smallest l all And a set of model parameter values ​​with the largest mIoU are exported and stored as a Pytorch-based .pt format file; S442: Use the torch.onnx.export function of the Pytorch library to save the .pt file as a .onnx format file; S443: Convert the .onnx format file to an engine and deploy it to the edge computing device; S444: In the edge computing device, reasoning is performed on the converted model based on the C++ environment, specifically including the following steps: S4441: The edge computing device reads the video stream captured by the visible light camera and divides it into image frames; S4442: The divided image frame is input into the inference model in the C++ environment, and the inference model outputs the prediction result of the image frame, and the prediction result includes the outline and type of the water leakage target in the image frame; S4443: The edge computing device synchronously reads the point cloud data obtained by the laser radar scanning, and spatially projects the point cloud according to the optical axis direction of the visible light camera based on the joint calibration matrix; S4444: Based on the plane pixel outline of the water leakage target, in the point cloud projection data, select the points within the water leakage outline for output. The output data is the position coordinates of the water leakage target in the real space, in the form of an array composed of several groups of three-dimensional space coordinates, expressed as: Cloud=[(x1,y1,z1),(x1,y1,z1),...,(x n ,y n ,z n )]; Among them, x, y, and z are the three-dimensional coordinates of the midpoint in space. The two-dimensional interpolation method is used to fit these coordinate values, and the shape of the leakage target in the three-dimensional space is quantitatively obtained. The integral method is used to quantitatively obtain the area of ​​the leakage shape surface.

8. The intelligent detection method for shield tunnel lining water leakage according to claim 7 is characterized by: In step S45, the pre-processing module pre-processes the visible light image, unifies the size and channel, and inputs it into the network; the prediction network module predicts and obtains the water leakage target information; The post-processing module outputs the quantitative parameters of the water leakage target based on the point cloud data; The post-processing module superimposes the visible light image, water leakage target mask and water leakage quantitative parameters and outputs them to the display device to realize real-time detection of tunnel lining water leakage targets.

Images