Earth height measuring method and device for earth chamber of shield machine, and shield machine

Abstract

The application discloses a kind of muck height measuring method and device of shield machine soil bin and shield machine, it is arranged on the inner wall of soil bin by pre-arranging height mark, height value character and M character arranged according to certain horizontal, longitudinal spacing, first with height mark, height value character and M character as mark, image acquisition is carried out after sample set is made, then training target detection model is carried out using sample set, then the height mark, height value character and M character in muck accumulation image are predicted and identified using the target detection model trained, finally, based on the prediction result, the height value of multiple muck height measuring points is intelligently calculated.The application measures muck height using machine vision technology, compared with the existing indirect measurement method of pressure, the soil bin muck height at different positions can be accurately measured, the whole measurement process is intelligent, without manual opening measurement, safe and reliable.

Description

Technical Field

[0001] The present invention relates to the technical field of shield machines, and in particular to a method and device for measuring the height of slag in a soil bin of a shield machine. In addition, the present invention also particularly relates to a shield machine using the device. Background Art

[0002] Shield construction methods are increasingly being used in the construction of long-distance, large-diameter tunnels due to their stability, safety, efficiency, and environmental friendliness. During tunnel boring, the shield machine relies on soil removed by the cutterhead to fill the soil bin, which supports the water and soil pressure of the excavation surface. The height of the soil accumulation in the bin directly affects the support pressure of the excavation surface. Furthermore, when the cutterhead is severely worn and needs to be opened to replace the cutter, the soil in the bin must be removed in advance. Before opening the bin, the bin is under pressure, and the hatch cannot be opened for observation at any time, making it difficult to accurately determine the height of the soil accumulation. Therefore, accurately measuring the height of the soil inside the bin is crucial for safe and efficient shield construction.

[0003] Currently, the height of the soil in the soil bin is mainly measured indirectly by measuring the earth pressure distribution inside the shield machine's soil bin. One conventional method uses the readings of pre-buried soil pressure gauges at different heights within the shield machine's soil bin. However, these pressure gauges are prone to inaccuracy due to long-term impact from the pressurized soil, resulting in poor measurement accuracy. Another conventional method involves adding monitoring holes in the upper half of the shield machine's bulkhead and connecting them to an external mechanical barometer. This method requires a dedicated operator to take pressure readings outside the shield machine's bulkhead, making it very inconvenient. Therefore, a method for measuring the height of soil in the soil bin with high measurement accuracy and ease of operation is urgently needed. Summary of the Invention

[0004] The present invention provides a method and device for measuring the height of slag in a soil bin of a shield machine, and a shield machine, so as to solve the technical problems of poor measurement accuracy and inconvenient operation in the existing soil bin slag height measurement method.

[0005] According to one aspect of the present invention, a method for measuring the height of slag in a soil bin of a shield machine is provided, comprising the following steps:

[0006] Collecting an image of a marked area on the inner wall of the soil bin, the marked area includes height marks, height value characters, and M characters arranged in advance on the inner wall of the soil bin at preset horizontal intervals. Multiple height marks, multiple height value characters, and multiple M characters are arranged at preset vertical intervals, and the height value characters, M characters, and height marks in the image are annotated to create a sample set.

[0007] Build a target detection model and use the sample set to train the target detection model until the model converges or reaches the preset accuracy;

[0008] Collect images of soil accumulation in the soil bin and use the trained object detection model to predict the height mark, height value character, and M character in the image;

[0009] Based on the prediction results of the height mark, height value character and M character in the muck accumulation image, the height values ​​of multiple muck height measurement points are calculated.

[0010] Furthermore, the process of calculating the height values ​​of the plurality of muck height measurement points based on the prediction results of the height mark, the height value character and the M character in the muck accumulation image is specifically as follows:

[0011] The height mark prediction box, height value character prediction box and M character prediction box belonging to the same vertical area are marked as a group of fused prediction boxes, and the fused mark area is determined according to the maximum and minimum coordinates of the vertices of each group of fused prediction boxes;

[0012] Binarize the highly marked areas in the fused marked areas and extract the connected areas;

[0013] The number of connected domains in the height mark area is counted, and the height value of the area blocked by the height mark is calculated by combining the height value character corresponding to its nearest horizontal neighbor M character and the vertical spacing between the height marks. Each area blocked by the height mark is a muck height measurement point.

[0014] Furthermore, the process of binarizing the highly marked areas in the fused marked areas and extracting connected areas is specifically as follows:

[0015] Divide the highly marked area into multiple grid blocks from top to bottom, and calculate the maximum and minimum grayscale values ​​of each grid block;

[0016] Perform global maximum and minimum filtering on the grayscale maximum and minimum values ​​calculated for all blocks, and use the maximum and minimum mean values ​​after filtering as the segmentation threshold for each grid block;

[0017] Each grid block is segmented based on the segmentation threshold to obtain a binary image of each grid block;

[0018] Connected domains are extracted from the binary image and their pixel areas are calculated. Connected domains with pixel areas smaller than a preset threshold are then removed.

[0019] Furthermore, before calculating the height value, the following is also included:

[0020] The minimum outer bounding rectangle of the maximum connected domain of the M character in the muck accumulation image is calculated, and the tilt angle of the minimum outer bounding rectangle is obtained. The muck accumulation image is tilt corrected based on the tilt angle.

[0021] Furthermore, the process of calculating the minimum outer bounding rectangle of the maximum connected domain of M characters in the muck accumulation image is specifically as follows:

[0022] The length and width of the prediction box of the M character are enlarged, and the M character is binarized using the global threshold method in the enlarged prediction box area to obtain a binary image of the M character;

[0023] Extract the outermost contour of the binary image and calculate the set of four contour vertices (upper left, lower left, upper right and lower right) in counterclockwise order according to the shape characteristics of the M character;

[0024] Starting from the upper left point, enumerate all the outer edges of the M character in counterclockwise order according to the vertex set;

[0025] For each outer edge enumerated, calculate the distance from the remaining points to the outer edge, and determine the point opposite the outer edge based on the maximum distance. Calculate the projection point of the remaining points to the outer edge to determine the length of the outer edge, and then calculate the area of ​​the rectangular frame of this enumeration based on the length of the outer edge and the maximum distance.

[0026] Repeat the above steps to calculate the area of ​​the rectangular box corresponding to each outer edge, and select the rectangular box with the smallest area as the minimum outer enclosing rectangle.

[0027] Furthermore, after the tilt correction is performed on the soil accumulation image, the following steps are also included:

[0028] The position of the prediction box is corrected based on the grayscale pixels of the prediction result to eliminate the prediction box deviation.

[0029] Furthermore, before collecting the image, the following steps are also included:

[0030] Check whether the lens is blurry. If so, clean the lens surface.

[0031] Furthermore, the process of detecting whether the lens is blurred is specifically as follows:

[0032] Set the blur threshold of the lens and perform smoothing filtering on the captured image;

[0033] Use the convolution kernel to perform convolution operation on the grayscale image to obtain a grayscale difference image;

[0034] Calculate the variance value of the grayscale difference image. If the variance value is less than the blur threshold, the lens is judged to be blurry. If the variance value is greater than or equal to the blur threshold, the lens is judged to be clear.

[0035] In addition, the present invention also provides a device for measuring the height of slag in a soil bin of a shield machine, which adopts the above-mentioned measuring method and comprises:

[0036] An image acquisition device is used to acquire images of the interior of the soil bin, including images of the marked area on the inner wall of the soil bin and images of the accumulated soil in the soil bin;

[0037] The host computer is used to mark the height value characters, M characters and height marks in the marked area image to produce a sample set, and after building the target detection model, use the sample set to train the target detection model until the model converges or reaches a preset accuracy, and then use the trained target detection model to predict the height marks, height value characters and M characters in the slag accumulation image. Then, based on the prediction results of the height marks, height value characters and M characters in the slag accumulation image, the height values ​​of multiple slag height measurement points are calculated.

[0038] In addition, the present invention also provides a shield machine that uses the above-mentioned slag height measuring device.

[0039] The present invention has the following effects:

[0040] The present invention provides a method for measuring the height of slag in a shield machine's earth bin. This method involves pre-arranging height markers, height value characters, and M characters on the inner wall of the earth bin at certain horizontal and vertical spacings. The method then uses the height markers, height value characters, and M characters as markers for image acquisition to create a sample set. The sample set is then used to train a target detection model. The trained target detection model is then used to predict and identify the height markers, height value characters, and M characters in the slag accumulation image. Finally, the height values ​​of multiple slag height measurement points are intelligently calculated based on the prediction results. The present invention utilizes machine vision technology for slag height measurement. Compared to existing indirect pressure measurement methods, the method can accurately measure the height of slag in the earth bin at different locations. The entire measurement process is intelligently performed, eliminating the need for manual bin opening for measurement and ensuring safety and reliability.

[0041] In addition, the device for measuring the height of slag in a soil bin of a shield machine and the shield machine of the present invention also have the above advantages.

[0042] In addition to the above-described objects, features and advantages, the present invention has other objects, features and advantages. The present invention will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are intended to explain the present invention and do not constitute an undue limitation of the present invention. In the accompanying drawings:

[0044] Figure 1 It is a flow chart of a method for measuring the height of slag in a soil bin of a shield machine according to a preferred embodiment of the present invention.

[0045] Figure 2It is a schematic diagram of a preset marking area on the inner wall of a soil bin in a preferred embodiment of the present invention.

[0046] Figure 3 Schematic diagram of the network structure of the underlying multi-scale feature interaction module of the preferred embodiment of the present invention.

[0047] Figure 4 yes Figure 1 Schematic diagram of the sub-process of step S4 in FIG.

[0048] Figure 5 yes Figure 4 Schematic diagram of the sub-process of step S42.

[0049] Figure 6 It is a schematic diagram of the principle of calculating the height value of the occluded area of ​​each height mark in a preferred embodiment of the present invention.

[0050] Figure 7 It is a flow chart of a method for measuring the height of slag in a soil bin of a shield machine according to another embodiment of the present invention.

[0051] Figure 8 yes Figure 7 Schematic diagram of the sub-process of step S34.

[0052] Figure 9 2 is a schematic diagram showing the principle of calculating the minimum outer enclosing rectangle of the M character in another embodiment of the present invention.

[0053] Figure 10 It is a flow chart of a method for measuring the height of slag in a soil bin of a shield machine according to another embodiment of the present invention.

[0054] Figure 11 It is a schematic diagram of the module structure of a device for measuring the height of slag in a soil bin of a shield machine according to another embodiment of the present invention. DETAILED DESCRIPTION

[0055] The embodiments of the present invention are described in detail below with reference to the accompanying drawings. However, the present invention can be implemented in many different ways as defined and covered below.

[0056] It is understandable that Figure 1 As shown, a preferred embodiment of the present invention provides a method for measuring the height of slag in a shield machine soil bin, comprising the following contents:

[0057] Step S1: Capturing an image of a marked area on the inner wall of a soil bin, the marked area includes height marks, height value characters, and M characters pre-placed on the inner wall of the soil bin at preset horizontal intervals. Multiple height marks, multiple height value characters, and multiple M characters are pre-placed at preset vertical intervals, and the height value characters, M characters, and height marks in the image are annotated to create a sample set.

[0058] Step S2: Build a target detection model and train it using the sample set until the model converges or reaches a preset accuracy;

[0059] Step S3: Collect an image of the soil accumulation in the soil bin, and use the trained object detection model to predict the height mark, height value character, and M character in the image;

[0060] Step S4: Based on the prediction results of the height mark, the height value character and the M character in the muck accumulation image, the height values ​​of the multiple muck height measurement points are calculated.

[0061] It can be understood that the method for measuring the height of the slag in the earth bin of a shield machine in this embodiment is to pre-arrange height marks, height value characters, and M characters arranged at certain horizontal and vertical intervals on the inner wall of the earth bin, first use the height marks, height value characters, and M characters as labels to collect images and then create a sample set, then use the sample set to train the target detection model, and then use the trained target detection model to predict and identify the height marks, height value characters, and M characters in the slag accumulation image, and finally, based on the prediction results, intelligently calculate the height values ​​of multiple slag height measurement points. The present invention uses machine vision technology to measure the height of slag. Compared with the existing indirect pressure measurement method, it can accurately measure the height of the slag in the earth bin at different positions. The entire measurement process is carried out intelligently, without the need for manual opening of the bin for measurement, and is safe and reliable.

[0062] It can be understood that in step S1, Figure 2 As shown, four groups of marked areas are pre-set on the inner wall of the soil bin. Each group of marked areas includes a height value character, an M character, and a height marker, arranged sequentially from left to right with a preset horizontal spacing. The height markers, height value characters, and M characters are spaced at a preset vertical spacing. Furthermore, a camera is mounted at approximately 3 / 4 of the bin's height using a pan / tilt platform to facilitate image acquisition. A light source is installed above the camera to provide illumination compensation. Due to the harsh imaging environment inside the bin, even with illumination compensation, some marked areas are not clearly captured by a color camera. Therefore, the present invention preferably uses an infrared camera to capture images of the bin interior. After capturing images of the marked areas, an image annotation tool, such as labelme, is used to annotate the height value characters, M characters, and height markers in the marked area images by category and location. A JSON file corresponding to each image is generated, which is then used to create the object detection sample set E. Then, using random, non-repeated sampling, 70%, 15%, and 15% of the sample set E are selected to form the training sample set E1, the validation sample set E2, and the test sample set E3.

[0063] Optionally, after creating the sample set E, the image sample data can be augmented using Fourier transform to improve the generalization of the model. Specifically, two images are randomly selected from the sample set E and Fourier transform is performed on each of them. The transformation formula is as follows:

[0064]

[0065]

[0066] Where fft() represents Fourier transform, x1 and x2 represent two images randomly selected from the sample set E, A1 and A2 represent the amplitudes of the two images respectively, and Then, take the two amplitudes A1 and A2, randomly generate a scale factor λ in the interval (0,1), and use λ and the complement of λ as weights to combine and obtain two new amplitudes. The combination formula is:

[0067] A1′=λA1+(1-λ)A2

[0068] A2′=λA2+(1-λ)A1

[0069] Here, A1′ and A2′ represent the two recombined amplitudes. Then, an inverse Fourier transform is performed based on the phase and recombined amplitudes to obtain two new images that retain the original outline but exchange contrast and color. These new images are then added to the sample set E.

[0070] It can be understood that in step S2, the YOLOv5 target detection model based on deep learning is built using the pytorch framework, and the multi-scale feature interaction module is constructed to enrich the multi-scale feature expression capability of the model. The structural diagram of the bottom multi-scale feature interaction module is shown in FIG. Figure 3As shown in the figure, the specific process of realizing the interactive fusion of multi-scale features is as follows: (1) Generate four feature maps of different scales P2, P3, P4, and P5 through the CSPDarknet-53 backbone network, and then use the convolution kernel of size 1*1*512 to uniformly adjust the number of channels of P2, P3, P4, and P5 to 512; (2) Use the nearest neighbor interpolation to downsample P2 and P3 to the size of P4 respectively, and use the maximum pooling window with stride 2 to upsample P5 to the size of P4, and then use the Concat operation to fuse the sampled feature maps to obtain a feature map of scale 40x40x512; (3) Use the depth-wise separable convolution of size 5*5*512 DW performs nonlinear feature fusion on the feature map to solve the semantic inconsistency and local position mismatch problems caused by upsampling and downsampling; (4) using the inverse operation of steps (1) and (2), the fused feature map is processed into feature maps P2′, P3′, P4′, P5′ of the same size as P2, P3, P4, and P5; (5) P2′ is fused with P2, P3′ with P3, P4′ with P4, and P5′ with P5 using the Concat operation to obtain feature maps of four scales: 160x160x128, 80x80x256, 40x40x512, and 20x20x1024, which are then fed into the subsequent FPN network part for target detection. It can be understood that the present invention embeds an embedded underlying multi-scale feature interaction module in the target detection model, enriches the multi-scale feature expression capability of the model, reduces the false detection rate of the mark, and can especially achieve high-precision recognition and positioning of highly marked features.

[0071] In addition, the present invention uses stochastic gradient descent method SGD combined with transfer learning technology to iteratively train the model offline until the model converges, and obtains a deep learning model containing the optimal weight ω. Specifically, the hyperparameters of the model are set first, and the initial learning rate lr is set to 1x10 -3 , the learning rate decay mode is set to exponential decay, the decay coefficient γ is set to 0.005, the batch size batch_size is set to 8, and the number of iterations epochs is set to 500; then, the input image is randomly rotated, translated, horizontally and vertically flipped according to the probability η, and the contrast, brightness and saturation of the image are randomly adjusted to achieve image enhancement and prevent overfitting in model training; the weight ω0 obtained by pre-training on the ImageNet public dataset is used to initialize the model parameters; then, the image size is scaled to 1280x1280x3 and sent to the model for offline iterative training. When epochs reaches 500 or the mAP accuracy of the model on the validation set reaches the specified value, the training is terminated to obtain the target detection model with the optimal weight ω, and then the model accuracy is tested on the test set.

[0072] It can be understood that in step S3, the image of the soil accumulation in the soil bin is collected by using an infrared camera and fed into the trained target detection model to predict the height mark, height value character and M character in the image, the category class, category confidence c and the coordinates of the upper left corner vertex of the prediction box bbox (x left ,y left ) and the coordinates of the lower right corner (x right ,y right ).

[0073] It is understandable that Figure 4 As shown, in step S4, the process of calculating the height values ​​of multiple muck height measurement points based on the prediction results of the height mark, height value character and M character in the muck accumulation image is specifically as follows:

[0074] Step S41: Mark the height mark prediction box, height value character prediction box and M character prediction box belonging to the same vertical area as a group of fused prediction boxes, and determine the fused mark area according to the maximum and minimum value coordinates of the vertices of each group of fused prediction boxes;

[0075] Step S42: binarizing the highly marked areas in the fused marked areas and extracting connected areas;

[0076] Step S43: Count the number of connected domains in the height mark area, and calculate the height value of the height mark blocked area in combination with the height value character corresponding to its nearest horizontal neighbor M character and the vertical spacing between the height marks. Each height mark blocked area is a slag height measurement point.

[0077] Specifically, in step S41, the height mark prediction frame, height value character prediction frame and M character prediction frame belonging to the same vertical area are marked as a group of fused prediction frames using the K-Means clustering algorithm according to the coordinates of the center point of the prediction frame, wherein the same vertical area refers to an area where the horizontal distance between the targets does not exceed a preset value. For example, Figure 2 The multiple prediction boxes corresponding to a set of marked areas in the image are integrated into a fused prediction box, which is equivalent to processing only a set of complete marked areas in the captured image. Then, since the height mark prediction box, height value character prediction box, and M character prediction box all contain a top left vertex coordinate and a bottom right vertex coordinate, the size of the fused prediction box is determined based on the maximum and minimum values ​​of the multiple top left vertices and multiple bottom right vertices, and the fused marked area corresponding to the fused prediction box is cropped into three parts: the height mark area, the height value character area, and the M character area.

[0078] It can be understood that when the camera takes an image, the image may contain more than one set of marked areas, and may also contain a portion of adjacent incomplete marked areas. In order to fully identify the marked area, the height mark prediction box, height value character prediction box and M character prediction box belonging to the same vertical area are first marked as a group of fused prediction boxes, and then the fused marked area is divided into three parts: height mark area, height value character area and M character area. This can remove the interference of irrelevant background and other marks adjacent to the current complete marked area, which is conducive to improving the calculation accuracy.

[0079] Then, in step S42, considering the influence of dark lighting and uneven lighting on image acquisition, the highly marked area is binarized using an adaptive threshold method and connected domains are extracted. Figure 5 As shown, the process of binarizing the highly marked areas in the fused marked areas and extracting connected areas is specifically as follows:

[0080] Step S421: Divide the height marked area into multiple grid blocks from top to bottom, and calculate the maximum and minimum grayscale values ​​of each grid block;

[0081] Step S422: Perform global maximum and minimum filtering on the grayscale maximum and minimum values ​​calculated for all blocks, and use the filtered maximum and minimum mean values ​​as the segmentation threshold for each grid block;

[0082] Step S423: Segment each grid block based on the segmentation threshold to obtain a binary image of each grid block;

[0083] Step S424: extracting connected domains from the binary image and calculating the pixel areas of the connected domains, and eliminating connected domains whose pixel areas are smaller than a preset threshold.

[0084] Specifically, the cropped height mark area is first divided into multiple grid blocks from top to bottom, for example, into 4 blocks, and the maximum and minimum grayscale values ​​of each grid block are calculated. Then, a global maximum and minimum value filtering process is performed on the maximum and minimum grayscale values ​​calculated for all blocks, and the maximum and minimum mean values ​​after filtering are used as the segmentation threshold T for each grid block. seg Then, based on the segmentation threshold T corresponding to each grid block seg The grid block is binarized to obtain a binary image of each grid block. Finally, a two-pass scanning method is used to extract the connected domains from the binary image of each grid block and calculate the pixel area S of each connected domain. i, and removes connected domains with pixel areas smaller than a preset threshold to eliminate the influence of noise data, which helps improve calculation accuracy. In addition, dilation and erosion operations can be used on the binary image of each grid block to eliminate small adhesions between markers of different heights and disconnect the connections between different markers, thereby optimizing the segmentation results.

[0085] It can be understood that in step S43, the numbers of cells in the same horizontal area H are counted in order from top to bottom. i The number of connected domains n i , that is, count the number of connected domains in each height mark from top to bottom, when the number of connected domains in the same horizontal area n i When it is 1, it means that the height mark here is not blocked, and when the number of connected domains in the same horizontal area is n i When it is greater than or equal to 2, it means that the height mark here is blocked at least once, and each height mark blocked area is a height measurement point. Figure 6 As shown, for each height mark occlusion area, the muck height value is calculated based on the following formula:

[0086] h i =kw*m

[0087] Among them, h i represents the height value of the area obscured by the height marker during the i-th statistical analysis, i.e., the soil height value. k represents the height value character corresponding to the nearest M character in the horizontal region, for example, 3 or 4. w represents the number of connected domains between the top of the M-marker prediction box and the obscured area of ​​the height marker. In this case, multiple connected domains in the same horizontal region are considered a single connected domain. m represents the vertical spacing between two adjacent height markers. Therefore, by calculating the height values ​​of the obscured areas of multiple height markers, multi-point measurement of the soil height in the silo can be achieved. The measurement accuracy is related to the actual vertical spacing of the height markers and can be adjusted according to actual needs. Furthermore, there is no need to pre-calibrate the correspondence between the height marker pixel coordinates and the true coordinates, which improves the accuracy and robustness of the soil height measurement.

[0088] In addition, in other embodiments of the present invention, before capturing an image, a cleaning device is controlled to automatically flush the surface of the marked area not covered by debris along a specified path to avoid missed or false detections in subsequent height mark recognition and character recognition due to obstruction by residual debris. In addition, after flushing, moisture or dirt may adhere to the camera lens, thereby affecting the image capture accuracy. Therefore, before capturing an image, the following steps are also included:

[0089] Check whether the lens is blurry. If so, clean the lens surface.

[0090] The present invention preferably adopts a lens blur detection algorithm to automatically detect whether the lens is blurry. Specifically, the lens blur threshold T is first set. obsure , and use a Gaussian kernel with a size of 3 and a variance of 0.8 to smooth the grayscale image captured by the infrared camera to reduce the impact of noise on detection. Then, use a 3×3 convolution kernel to perform convolution operation on the grayscale image captured by the infrared camera to obtain a grayscale difference image, where the convolution kernel parameters are [0,-1,0,-1,4,-1,0,-1,0]. Then calculate the variance value σ of the grayscale difference image. If the variance value σ is less than the fuzziness threshold T obsure , then the lens is judged to be blurred, if the variance value σ is greater than or equal to the blur threshold T obsure , and then judge whether the lens is clear. When the lens is detected to be blurry, it means that there is water vapor or dirt attached to the lens surface, and then the air cleaning device is controlled to automatically clean the lens surface.

[0091] Alternatively, in another embodiment of the present invention, Figure 7 As shown, the muck height measurement method further includes the following steps before calculating the height value:

[0092] Step S34: Calculate the minimum outer enclosing rectangle of the maximum connected domain of the M character in the muck accumulation image, obtain the tilt angle of the minimum outer enclosing rectangle, and perform tilt correction on the muck accumulation image based on the tilt angle.

[0093] Among them, Figure 8 As shown in FIG, the process of calculating the minimum outer bounding rectangle of the maximum connected domain of M characters in the muck accumulation image is specifically as follows:

[0094] Step S341: Enlarging the length and width of the prediction box of the character M, and binarizing the enlarged prediction box area using a global threshold method to obtain a binary image of the character M;

[0095] Step S342: extracting the outermost contour of the binary image, and obtaining the set of four contour vertices, namely, the upper left, lower left, upper right, and lower right, in a counterclockwise order according to the shape characteristics of the M character;

[0096] Step S343: Starting from the upper left point, enumerate all the outer edges of the M character in counterclockwise order according to the vertex set;

[0097] Step S344: For each outer edge enumerated, calculate the distance from the remaining points to the outer edge, and determine the point opposite the outer edge based on the maximum distance. Calculate the projection point of the remaining points onto the outer edge to determine the length of the outer edge, and then calculate the area of ​​the rectangular frame enumerated this time based on the length of the outer edge and the maximum distance.

[0098] Step S345: Repeat the above step S344 to calculate the area of ​​the rectangular frame corresponding to each outer edge, and select the rectangular frame with the smallest area as the minimum outer enclosing rectangle.

[0099] Specifically, the length and width of the prediction box of the character M are enlarged by 0.2 times, and the enlarged prediction box area is binarized using the global threshold method to obtain the binary image of the character M. Then, the outermost contour of the binary image of the character M is extracted, and the set P of the four contour vertices of the upper left, lower left, upper right and lower right is obtained in counterclockwise order according to the shape characteristics of the character M, where P = {P1, P2, P3, P4}, as shown in the following example: Figure 9 As shown. Starting from the upper left point P1, all the outer edges of the character M are enumerated in counterclockwise order according to the vertex set P, namely P1P2, P2P3, P3P4, and P4P1. For each outer edge enumerated, calculate the distance from the remaining points to the outer edge, and determine the point opposite the outer edge based on the maximum distance, calculate the projection point of the remaining points to the outer edge to determine the length of the outer edge, and then calculate the area of ​​the rectangular frame enumerated this time based on the length of the outer edge and the maximum distance. For example, for the outer edge P1P2, determine the opposite point as P4 based on the maximum distance, and then calculate the projection point of P3 to the outer edge P1P2 to determine the length of the outer edge P1P2. Based on the length of the outer edge P1P2 and the maximum distance Distance, the area of ​​the rectangular frame this time S1 can be calculated. Repeat the above steps to calculate the area of ​​the rectangular frame corresponding to each outer edge, and filter out the rectangular frame S with the smallest area. i as the minimum outer bounding rectangle.

[0100] It is understandable that because the infrared camera is mounted on a gimbal to capture images from different perspectives, it cannot guarantee that the height marker and the M character will be horizontal in the image during each capture. When the height marker and the M character are not horizontally aligned, calculating the number of connected domains in the height marker area may lead to statistical errors. Therefore, the present invention calculates the minimum outer bounding rectangle of the maximum connected domain of the M character and uses the tilt angle of the minimum outer bounding rectangle to perform tilt correction on the image, ensuring the horizontal alignment of the height marker and the M character, thereby greatly reducing the statistical error of the connected domain in the height marker area and improving the overall calculation accuracy.

[0101] In addition, the prediction box in the trained target detection model is likely to surround the target, but after the tilt correction in step S34, since only the target rotates but the prediction box does not, the target may cross the boundary and exceed the range of the prediction box. In particular, there is a problem of prediction box deviation between the prediction box and the actual position of the height mark, which reduces the measurement accuracy. Figure 10 As shown, preferably, the method for measuring the height of slag soil further includes the following contents after performing tilt correction on the slag soil accumulation image:

[0102] Step S34a: Correct the position of the prediction frame based on the grayscale pixels of the prediction result to eliminate the prediction frame deviation.

[0103] Specifically, traverse the grayscale image pixels within the height mark prediction box, where the total number of pixels is S p , sort all pixel values ​​in ascending order, set the proportional coefficient α, the value of α is between [0.6, 0.75], and select the α*S in the sequence p The pixel value V of the pixel point p As the pixel value threshold V t .

[0104] Then, traverse the pixel points on the upper border AB of the height mark prediction box. If no pixel value on the upper border AB is greater than V t , which means that the upper border AB is outside the outermost contour of the target, then the upper border moves down by 1 pixel, and continues to traverse the new upper border. If it is still not greater than V t , the upper border continues to move down by 1 pixel until the pixel value of a pixel on the upper border is greater than V t , record the endpoints A1 and B1 of the new upper border. Then, move the new upper border down by 1 pixel 4 times in a row, and check whether the pixel values ​​on the upper border are all greater than V t If the pixel point exists, A1 and B1 are the endpoints of the new upper border. If not, the upper border continues to move down by 1 pixel and then traverses the pixel value 4 times until the above requirements are met. At this time, the endpoints A1 and B1 of the new upper border are recorded. In addition, if the pixel value of the pixel point on the upper border AB is greater than V t , which means that the upper border AB is inside the outermost contour of the target, then the upper border moves up by 1 pixel, and continues to traverse the new upper border. If there is still a pixel value greater than V t , the upper border continues to move upward by 1 pixel until all pixel values ​​on the upper border are less than V t , record the endpoints A1 and B1 of the new upper border. At this point, the correction of the upper border is completed.

[0105] Repeat the above steps, correcting the right border (B1-C) to obtain the new border (B2-C1); correcting the bottom border (C1-D) to obtain the new border (C2-D1); and correcting the left border (D1-A1) to obtain the new border (D2-A2). At this point, the height marker predicted border is corrected, and the four vertices of the new border are (A2, B2, C2, D2).

[0106] In addition, if Figure 11As shown, another embodiment of the present invention further provides a device for measuring the height of slag in a soil bin of a shield machine, preferably using the measurement method described above, including:

[0107] An image acquisition device is used to acquire images of the interior of the soil bin, including images of the marked area on the inner wall of the soil bin and images of the accumulated soil in the soil bin;

[0108] The host computer is used to annotate the height value characters, M characters, and height markers in the marked area image to create a sample set. After constructing a target detection model, the sample set is used to train the target detection model until the model converges or reaches a preset accuracy. The trained target detection model is then used to predict the height markers, height value characters, and M characters in the muck accumulation image. Based on the predicted height markers, height value characters, and M characters in the muck accumulation image, the height values ​​of multiple muck height measurement points are calculated. Preferably, an infrared camera is used as the image acquisition device, and an industrial computer is preferably used as the host computer.

[0109] It can be understood that the device for measuring the height of the slag in the earth bin of a shield machine in this embodiment pre-arranges height marks, height value characters, and M characters arranged at certain horizontal and vertical intervals on the inner wall of the earth bin, first uses the height marks, height value characters, and M characters as labels for image acquisition to create a sample set, then uses the sample set to train a target detection model, and then uses the trained target detection model to predict and identify the height marks, height value characters, and M characters in the slag accumulation image. Finally, based on the prediction results, the height values ​​of multiple slag height measurement points are intelligently calculated. The present invention uses machine vision technology to measure the height of slag. Compared with the existing indirect pressure measurement method, it can accurately measure the height of the slag in the earth bin at different positions. The entire measurement process is carried out intelligently, without the need for manual opening of the bin for measurement, and is safe and reliable.

[0110] In addition, another embodiment of the present invention further provides a shield machine, which preferably adopts the above-mentioned slag height measuring device.

[0111] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A method for measuring the height of slag in a shield machine soil bin, characterized in that: Includes the following: Collecting an image of a marked area on the inner wall of the soil bin, the marked area includes height marks, height value characters, and M characters arranged in advance on the inner wall of the soil bin at preset horizontal intervals. Multiple height marks, multiple height value characters, and multiple M characters are arranged at preset vertical intervals, and the height value characters, M characters, and height marks in the image are annotated to create a sample set. Build a target detection model and use the sample set to train the target detection model until the model converges or reaches the preset accuracy; Collect images of soil accumulation in the soil bin and use the trained object detection model to predict the height mark, height value character, and M character in the image; Based on the prediction results of the height mark, height value character and M character in the muck accumulation image, the height values ​​of multiple muck height measurement points are calculated; The process of calculating the height values ​​of the plurality of muck height measurement points based on the prediction results of the height mark, the height value character and the M character in the muck accumulation image is specifically as follows: The height mark prediction box, height value character prediction box and M character prediction box belonging to the same vertical area are marked as a group of fused prediction boxes, and the fused mark area is determined according to the maximum and minimum coordinates of the vertices of each group of fused prediction boxes; Binarize the highly marked areas in the fused marked areas and extract the connected areas; The number of connected domains in the height mark area is counted, and the height value of the area blocked by the height mark is calculated by combining the height value character corresponding to its nearest horizontal neighbor M character and the vertical spacing between the height marks. Each area blocked by the height mark is a soil height measurement point. The process of binarizing the highly marked areas in the fused marked areas and extracting connected areas is specifically as follows: Divide the highly marked area into multiple grid blocks from top to bottom, and calculate the maximum and minimum grayscale values ​​of each grid block; Perform global maximum and minimum filtering on the grayscale maximum and minimum values ​​calculated for all blocks, and use the maximum and minimum mean values ​​after filtering as the segmentation threshold for each grid block; Each grid block is segmented based on the segmentation threshold to obtain a binary image of each grid block; Extract connected domains from the binary image and calculate the pixel area of ​​the connected domains, and remove connected domains whose pixel area is less than a preset threshold; Also include the following before calculating the height value: The minimum outer bounding rectangle of the maximum connected domain of the M character in the muck accumulation image is calculated, and the tilt angle of the minimum outer bounding rectangle is obtained. The muck accumulation image is tilt corrected based on the tilt angle.

2. The method for measuring the height of the slag in the soil bin of a shield machine according to claim 1, characterized in that: The specific process of calculating the minimum outer bounding rectangle of the maximum connected domain of M characters in the muck accumulation image is as follows: The length and width of the prediction box of the M character are enlarged, and the M character is binarized using the global threshold method in the enlarged prediction box area to obtain a binary image of the M character; Extract the outermost contour of the binary image and calculate the set of four contour vertices (upper left, lower left, upper right and lower right) in counterclockwise order according to the shape characteristics of the M character; Starting from the upper left point, enumerate all the outer edges of the M character in counterclockwise order according to the vertex set; For each outer edge enumerated, calculate the distance from the remaining points to the outer edge, and determine the point opposite the outer edge based on the maximum distance. Calculate the projection point of the remaining points to the outer edge to determine the length of the outer edge, and then calculate the area of ​​the rectangular frame of this enumeration based on the length of the outer edge and the maximum distance. Repeat the above steps to calculate the area of ​​the rectangular box corresponding to each outer edge, and select the rectangular box with the smallest area as the minimum outer enclosing rectangle.

3. The method for measuring the height of the slag in the soil bin of a shield machine according to claim 1, wherein: After the slope correction of the soil accumulation image is performed, the following contents are also included: The position of the prediction box is corrected based on the grayscale pixels of the prediction result to eliminate the prediction box deviation.

4. The method for measuring the height of the slag in the earth bunker of a shield machine according to claim 1, wherein: Before acquiring the image, the following also applies: Check whether the lens is blurry. If so, clean the lens surface.

5. The method for measuring the height of the slag in the earth bunker of a shield machine according to claim 4, characterized in that: The process of detecting whether the lens is blurred is specifically as follows: Set the blur threshold of the lens and perform smoothing filtering on the captured image; Use the convolution kernel to perform convolution operation on the grayscale image to obtain a grayscale difference image; Calculate the variance of the grayscale difference image. If the variance is less than the blur threshold, the lens is considered blurry. If the variance is greater than or equal to the blur threshold, the lens is considered clear.

6. A device for measuring the height of slag in a shield machine soil bin, using the measuring method according to any one of claims 1 to 5, characterized in that: include: An image acquisition device is used to acquire images of the interior of the soil bin, including images of the marked area on the inner wall of the soil bin and images of the accumulated soil in the soil bin; The host computer is used to mark the height value characters, M characters and height marks in the marked area image to produce a sample set, and after building the target detection model, use the sample set to train the target detection model until the model converges or reaches a preset accuracy, and then use the trained target detection model to predict the height marks, height value characters and M characters in the slag accumulation image. Then, based on the prediction results of the height marks, height value characters and M characters in the slag accumulation image, the height values ​​of multiple slag height measurement points are calculated.

7. A shield machine, characterized in that: The slag height measuring device as described in claim 6 is used.

Images