TBM guniting quality real-time monitoring and control system and method based on visual servo

Abstract

The invention provides a TBM guniting quality real-time monitoring and control system based on visual servo, and the system comprises a visual collection unit which is used for collecting the image information of a guniting working surface in real time; the motion execution unit is provided with a guniting tail end executor and is used for guniting the guniting working surface; the feeding execution unit is used for conveying various kinds of slurry to the slurry spraying end executor; the intelligent control unit is used for performing data analysis on the image information acquired by the visual acquisition unit, comparing the image information with a target image information threshold value, and controlling the guniting end effector and the feeding execution unit to execute corresponding actions; meanwhile, the invention further discloses a control method of the TBM guniting quality real-time monitoring and control system based on the visual servo. According to the invention, real-time visual information acquisition is carried out on the guniting position, closed-loop control is realized by comparing image information such as gray scale with image features of target guniting, unmanned operation of the guniting process is realized, and flatness and consistency of the whole guniting area can be ensured.

Description

Technical Field

[0001] This invention relates to the technical field of intelligent shotcrete, and in particular to a real-time monitoring and control system and method for shotcrete quality of TBM based on vision servoing. Background Technology

[0002] With the development of tunnel construction technology, TBM (Tunnel Boring Machine) construction has become a common tunnel construction method. Shotcrete operation is a crucial initial support procedure after TBM excavation and a vital link in ensuring tunnel construction safety. Currently, the shotcrete operation process in TBM tunnels can include the following steps: First, the tunnel is excavated, exposing the surface rock. The shotcrete equipment moves with the tunnel boring machine to the surface to be shotcreted. A person manually assesses the over- or under-excavation status of the surface and uses a handheld remote control to activate the shotcrete system's concrete pump, accelerator pump, air compressor, etc. The remote control also controls the shotcrete robotic arm to perform corresponding movements to ensure the shotcrete completely covers the surface and achieves the specified thickness and smoothness. All of these operations are performed manually using a handheld remote control. However, due to limitations in observation ability, reaction time, skill level, and concrete rebound, it is difficult for operators to achieve the required shotcrete thickness and uniformity. Furthermore, operators are exposed to high levels of shotcrete dust for extended periods, posing significant health and safety hazards. Therefore, there is an urgent need to develop intelligent shotcrete operation systems to achieve unmanned operation of the shotcrete process.

[0003] In recent years, with the deepening research on shotcrete systems and feature recognition, devices and methods for achieving intelligent shotcrete operations using intelligent technologies have gradually emerged. For example, the invention patent with authorization announcement number CN110159313B, entitled "An Intelligent Shotcrete System and its Shotcrete Support Method," uses image data obtained from 3D scanning to output the rotation speed of a robotic arm, controls the nozzles to spray shotcrete into the tunnel, and scans again after spraying to compare the results and proceed with the next span of shotcrete operation. Another invention patent with authorization announcement number CN117773918B, entitled "A Shotcrete Path Planning Method for an Intelligent Shotcrete Robot Based on Point Cloud Processing," uses a lidar mounted on a shotcrete robot to acquire tunnel point clouds and performs point cloud processing on these clouds. This process generates a spraying path and guides the spraying robot to perform its work. A patent with authorization announcement number CN114924555B, entitled "A Path Planning Method Based on a Fully Automated Tunnel Spraying Robot," establishes a static spraying growth model related to the spraying machine, spraying time, and spraying distance. The spraying process simulates the movement of this static model, and different superposition results are obtained based on the movement speed. With a target final thickness, the machine's path is planned to achieve the desired spraying thickness while ensuring a relatively smooth surface. Based on this patent, currently disclosed TBM intelligent spraying devices and control methods are based on first extracting point cloud features from the surface to be sprayed using a scanner. Based on these features and the required spraying thickness at each location, a spraying path is planned, and unmanned spraying operations are executed. This method represents a theoretical model of intelligent spraying and is difficult to implement in actual operation. Because the state of the slurry during shotcreting and the rebound amount at different shotcreting locations cannot be measured and evaluated, the pre-planned shotcreting path and shotcreting time cannot match the path and time required to achieve the target shotcreting quality. In other words, this solution cannot perceive the shotcreting quality in real time for closed-loop control. At the same time, due to the large amount of 3D point cloud data to process, the data processing speed is difficult to match with the movement speed of the shotcreting robotic arm, making servo control difficult to implement. Summary of the Invention

[0004] To address the technical problems in existing shotcrete technology, such as the inability of point cloud recognition methods to perceive shotcrete quality in real time for closed-loop control, and the mismatch between point cloud data processing delay and motion control, this invention proposes a TBM shotcrete quality real-time monitoring and control system and method based on visual servoing. By acquiring real-time visual information of the shotcrete position, closed-loop control is achieved by comparing grayscale and other image information with the image features of the target shotcrete, enabling unmanned operation of the shotcrete process while ensuring the flatness and consistency of the entire shotcrete area.

[0005] To achieve the above objectives, the technical solution of the present invention is as follows: a real-time monitoring and control system for TBM shotcrete quality based on visual servoing, comprising:

[0006] The vision acquisition unit is used to acquire image information of the shotcrete working surface in real time.

[0007] The motion execution unit is equipped with a shotcrete end effector to perform shotcrete spraying on the shotcrete working surface;

[0008] The feeding unit delivers various types of slurry to the shotcrete end effector.

[0009] The intelligent control unit analyzes the image information acquired by the vision acquisition unit, compares it with the target image information threshold, and controls the shotcrete end effector and the material feeding execution unit to perform corresponding actions.

[0010] The material feeding execution unit is connected to the shotcrete end effector, and the vision acquisition unit, motion execution unit, and material feeding execution unit are all connected to the intelligent control unit.

[0011] Preferably, both the vision acquisition unit and the shotcrete end effector are movably mounted on the rotary platform, which is slidably mounted on the traveling platform; the traveling platform drives the rotary platform to move horizontally, thereby adjusting its distance from the shotcrete area.

[0012] The vision acquisition unit integrates a multispectral industrial camera, including but not limited to RGB cameras and infrared cameras. An anti-adhesion coating is applied to the glass in front of the industrial camera, enabling stable and reliable acquisition of image data.

[0013] Preferably, the motion execution unit is a work servo execution mechanism with a multi-degree-of-freedom intelligent shotcrete robotic arm. The end of the work servo execution mechanism is equipped with a shotcrete end effector. The work servo execution mechanism realizes the forward, backward, back-and-forth swing, left-and-right swing, and brushing motion of the shotcrete end effector.

[0014] The feeding execution unit includes multiple slurry conveying systems, with pressure sensors and flow sensors installed on each system; thereby controlling the stability of slurry conveying based on the collected pressure and flow data.

[0015] Preferably, the intelligent control unit includes a processor and a control module. The vision acquisition unit, pressure sensor, and flow sensor are all connected to the processor. The processor is connected to the control module. The control module is connected to the work servo actuator, the shotcrete end effector, and the slurry conveying system, respectively.

[0016] Preferably, the visual acquisition unit includes a first visual acquisition unit and a second visual acquisition unit, and the shotcrete end effector is a nozzle assembly. The first and second visual acquisition units are respectively located on both sides of the nozzle assembly. When the nozzle assembly is performing shotcrete operation on the left side, the first visual acquisition unit on the right side acquires image information; when the nozzle assembly is performing shotcrete operation on the right side, the second visual acquisition unit on the left side acquires image information. Data acquisition on both sides can protect the corresponding visual acquisition units.

[0017] Preferably, the first and second vision acquisition units extend out and are on the same cross-section as the nozzle assembly when in operation;

[0018] The nozzle assembly or rotating platform is equipped with a supplementary lighting array for the first and second vision acquisition units. The nozzle assembly is used to spray grout into the designated spraying area.

[0019] Preferably, both the first and second visual acquisition units include a protective chamber, inside which a camera is housed. A first water rinsing assembly and an air rinsing assembly are fixedly installed on the outer circumference of the camera. The camera, the first water rinsing assembly, and the air rinsing assembly are all mounted on a fixed base, which is connected to a telescopic hydraulic cylinder. The telescopic hydraulic cylinder is located at one end of the protective chamber, and its cylinder body is located outside the protective chamber. The protective chamber protects the camera when it is not in use, and the water rinsing and air curtain ensure that the glass cover on the front of the camera is clean during operation, without affecting image acquisition.

[0020] A second water flushing assembly is installed on the circumference of the front part of the protective chamber for flushing the camera and the cylinder of the telescopic hydraulic cylinder when it retracts or extends.

[0021] Preferably, the protective chamber and the cylinder body of the telescopic cylinder are fixedly mounted on the rotary gear ring of the rotary platform, and a rotary drive unit is fixed inside the rotary gear ring, which drives the rotary gear ring to rotate.

[0022] The camera is a monocular camera, a binocular camera, or a combination of a camera and an infrared camera;

[0023] The nozzle assembly is fixed on the rotary gear ring of the rotary platform. The position of the nozzle assembly relative to the tunnel is determined by reading the angle of the encoder of the rotary drive unit of the rotary platform. The position of the vision acquisition unit is determined by measuring the rotation of the rotary drive unit.

[0024] Preferably, the nozzle assembly includes a nozzle mixer, which is connected to a left-right swing drive and a front-back swing drive respectively. The left-right swing drive and the front-back swing drive are connected in series. Both the left-right swing drive and the front-back swing drive are fixed on the rotary gear ring of the rotary platform by a fixed base. The nozzle mixer is connected to the slurry conveying system.

[0025] Preferably, the slurry conveying system includes an air compressor, a concrete conveying pump, and a quick-setting agent pump, both of which are connected to the air compressor, and the air compressor is connected to the nozzle mixer.

[0026] Preferably, the front ends of the first water flushing assembly, the air flushing assembly, and the second water flushing assembly are all equipped with solenoid valves, which are connected to the control module; the nozzles of the first water flushing assembly and the second water flushing assembly are connected to the high-pressure water pump through solenoid valves, and the nozzles of the air flushing assembly are connected to the air compressor through solenoid valves.

[0027] Preferably, during the retraction of the visual acquisition unit, the retraction action is interlocked with the second water flushing component until it retracts to the correct position; when the visual acquisition unit extends, the extension and retraction action is interlocked with the first water flushing component until it extends to the correct position; the first water flushing component is turned off, and the air flushing component is turned on to flush away the water droplets remaining on the camera lens cover; during the shotcreting operation, the air flushing component is normally open to form an air curtain protection, while the first water flushing component is turned on at set time intervals for flushing.

[0028] A control method for a real-time monitoring and control system for TBM shotcrete quality based on vision servoing is described. During tunnel shotcrete support, the air compressor, concrete pump, and accelerator pump of the grout delivery system are first started to complete pre-shotcrete preparation. After the shotcrete operation begins, the first and second vision acquisition units start working, automatically acquiring the position of the nozzle assembly using the angle measured by the encoder of the rotary drive unit. When performing shotcrete operation on the left side, the camera of the first vision acquisition unit on the right extends from its protective chamber to a position suitable for observing the shotcrete status, while the second vision acquisition unit on the left retracts from its protective chamber. The reverse is repeated when performing shotcrete operation on the right side. During the extension and retraction... The second water flushing component opens to flush the camera with water. During the shotcreting operation, the air flushing component on the vision acquisition unit that acquires images sprays air to form an air curtain for protection. At the same time, after reaching the set interval, the first water flushing component is activated to flush and clean the camera's outer casing. During the process of the vision acquisition unit retracting into the protective chamber, the second water flushing component arranged on the protective chamber flushes the cylinder rod of the telescopic hydraulic cylinder and the camera. The image information acquired by the vision acquisition unit is transmitted to the intelligent control unit via video or image, marking the target area of ​​the shotcreting operation in the image information and extracting feature information within the target area. The feature information is used to determine whether the shotcreting quality meets the standards.

[0029] Preferably, the intelligent control unit establishes an image grayscale threshold G1 when the shotcrete quality meets the standard based on the grayscale of the feature information. The image grayscale threshold G1 is compared with the real-time image grayscale value ΔG in real time. When ΔG≠G1, the residence time of the nozzle mixer, the nozzle angle, the real-time flow rate of the concrete delivery pump, the real-time flow rate of the accelerator pump, and the pressure of the air compressor are dynamically adjusted until ΔG=G1. Then, the operation servo actuator of the shotcrete robotic arm is moved to the adjacent non-compliant area to repeat the above shotcrete steps and complete the shotcrete operation in the work area.

[0030] Preferably, the dynamic adjustment is made according to the mapping relationship |ΔG-G1|=|T, α, β, K, X, S|, where T is the shotcrete dwell time, α is the left and right swing angle of the nozzle, β is the front and back swing angle of the nozzle, K is the real-time flow rate of the concrete delivery pump, X is the real-time flow rate of the accelerator pump, and S is the pressure of the air compressor.

[0031] Preferably, the image grayscale threshold G1 for achieving the required shotcrete quality is determined by combining the shotcrete quality standard and the on-site environment. The implementation method is as follows:

[0032] Image sample collection: For locations deemed "qualified for shotcreting" by humans, use the same visual acquisition unit as the site and under the same lighting conditions to collect samples of the tunnel surface after shotcreting, and select the areas with qualified shotcreting quality in the image annotation tool and mark them as "qualified areas"; collect no less than 500 image samples.

[0033] Image grayscale feature extraction: Convert the RGB images in the image samples into grayscale values ​​using a weighted average method;

[0034] The arithmetic mean of the grayscale values ​​of all image samples from the acquired "qualified areas" is calculated and recorded as the grayscale mean. ;

[0035] The initial optimal image threshold is set using the formula for calculating inter-class variance;

[0036] The optimal image threshold is dynamically corrected based on the actual tunnel environment to obtain the image grayscale threshold G1 that meets the shotcrete quality standard.

[0037] Preferably, a filtering method is used to remove noise interference caused by dust and slurry splashing in the spraying image before extracting the grayscale features of the image;

[0038] The method for determining the initial optimal image threshold is as follows: Image samples from a specific spraying process are collected, and regions with grayscale values ​​≥ k are designated as "qualified regions." The proportion of pixels in the "qualified regions" to the total image size is... The average grayscale value of the pixels in the "qualified area" is The region with a grayscale value < k is designated as the "unacceptable region," and its pixel count accounts for a certain percentage of the total image. The average grayscale value of the pixel is ;k represents the grayscale threshold, ranging from 0 to 255;

[0039] Using the formula for calculating between-class variance, the between-class variance is calculated as follows:

[0040] = ;

[0041] Iterate through the grayscale value thresholds k from 0 to 255, and calculate the inter-class variance for each grayscale value threshold k. Find the variance between classes The value of the grayscale threshold k at the maximum value is the initial optimal image threshold. .

[0042] Preferably, the method for dynamically correcting the optimal image threshold is as follows: identifying the light intensity and dust concentration at the construction site, and obtaining the current ambient brightness through a light sensor. The mass concentration of dust in the air is obtained through dust sensors inside the tunnel. ; for the initial optimal image threshold The image grayscale threshold for achieving the final shotcrete quality standard is dynamically determined through correction. for:

[0043]

[0044] , The optimal image thresholds at the initial calibration are respectively The light intensity and dust concentration were recorded at the time. , These are the compensation coefficients for illumination intensity and dust concentration, respectively.

[0045] Preferably, the real-time grayscale value of the image within the spraying range is calculated based on the real-time image information acquired during the spraying process. Image grayscale threshold The grayscale deviation characteristics between them are used to guide the various actuators of the intelligent shotcrete system to make corresponding time and pose adjustments based on the neural network.

[0046] Preferably, the method for calculating the grayscale deviation feature is as follows:

[0047] Calculate the absolute deviation: ;

[0048] Calculate the relative deviation: And relative deviation It is a slight deviation. The deviation is moderate. There is a slight height deviation;

[0049] Deviation change rate: ,in, The sampling frequency; This represents the absolute grayscale deviation value related to the shotcrete area at time t. This represents the absolute grayscale deviation value related to the shotcrete area at time t-1;

[0050] Gray variance: , This represents the number of pixels in the sprayed area. , This represents the grayscale value of the i-th pixel within the sprayed area;

[0051] Based on construction experience: when the grayscale variance Uneven grayscale distribution in the shotcrete area indicates insufficient shotcrete thickness; when the grayscale variance... This indicates that the gray distribution in the sprayed area is uniform, indicating that the sprayed thickness is sufficient.

[0052] Preferably, the neural network achieves the mapping from image feature information to the parameters of each actuator in the jetting process through an "input-feature extraction-output" signal transmission process; the neural network includes an input layer, hidden layer I, hidden layer II, and an output layer connected in sequence; wherein, the input layer includes 4 neurons, corresponding to 4 gray-level deviation features: ;in, , , , ;

[0053] Each neuron in hidden layer I performs a weighted sum of the input parameters, and then... Activation functions filter out invalid information;

[0054] Hidden layer II performs a second weighted summation on the 16 low-order features output by hidden layer I to generate high-order features;

[0055] Output layer: Hidden layer II outputs 12 higher-order features, which are directly related to the output parameters and are then processed by a final weighted summation and activation function. Obtain the execution parameters.

[0056] Preferably, the weighted summation method for hidden layer I is as follows: the first... The input to each neuron is:

[0057]

[0058] It is the first of the input layer Parameters To the first hidden layer I The weights of each neuron, ; It is a bias term;

[0059] The activation function of the hidden layer I for ;

[0060] The weighted summation method for hidden layer II is as follows: the first... The input to each neuron is:

[0061]

[0062] It is the first hidden layer I The first neuron to hidden layer II The weights of each neuron, ; It is a bias term;

[0063] Hidden Layer II The activation function filters out invalid information as follows: ;

[0064] The weighted summation of the output layer is: the first... Input to each neuron:

[0065]

[0066] in, It is the second hidden layer II The first neuron to the output layer The weights of each neuron, ; For bias terms;

[0067] The method for obtaining the execution parameters is as follows:

[0068]

[0069]

[0070] in, For activation function, These are the actual shotcrete parameters, and Residence time of the nozzle , The left and right swing angle of the nozzle assembly , The forward and backward swing angle of the nozzle assembly , Real-time flow rate of shotcrete delivery pump , Real-time flow rate of accelerator pump , air compressor pressure ; , These are the maximum and minimum thresholds for the corresponding parameters. These represent intermediate parameters in the calculation, which are the actual output parameters. Previous forecast values.

[0071] Preferably, the weight Weight and weight The bias term is randomly initialized during training. Bias terms and bias terms Initialize to 0;

[0072] The neural network model was trained using data collected on-site. Simultaneously, the weights of hidden layer I, hidden layer II, and the output layer were redistributed, and a new round of training was conducted, utilizing the loss function. To conduct an evaluation;

[0073] When the loss function The model training is complete at 5%.

[0074] Compared with existing technologies, the advantages of this invention are as follows: This invention no longer calculates the spraying time and path required for the sprayed area to reach a certain thickness. Instead, it acquires real-time visual information of the spraying position and compares it with the target spraying image features using grayscale and other image information. Once the features completely overlap with the target features, it moves to an adjacent position with different features to perform the spraying operation until the entire tunnel cross-section is sprayed, achieving unmanned operation of the spraying process while ensuring the flatness and consistency of the entire sprayed area. This invention enables millisecond-level real-time feedback of feature data, solving the problem in existing spraying technologies where point cloud recognition schemes cannot perceive spraying quality in real time for closed-loop control, and also addressing the mismatch between point cloud data processing latency and motion control. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1 This is a schematic diagram of the system of the present invention.

[0077] Figure 2 This is a schematic diagram showing the cooperative state of the rotary platform and the traveling platform of the present invention.

[0078] Figure 3 This is a structural diagram of the nozzle assembly and vision acquisition unit of the present invention.

[0079] Figure 4 For the present invention Figure 3 A magnified view of a portion of the visual acquisition unit.

[0080] Figure 5 This is a flowchart of the control method of the present invention.

[0081] In the diagram, 1 is the first visual acquisition unit; 2 is the second visual acquisition unit; 3 is the nozzle assembly; 4 is the rotating platform; 5 is the traveling platform; 6 is the camera; 7 is the first water flushing assembly; 8 is the air flushing assembly; 9 is the second water flushing assembly; 10 is the telescopic cylinder; 11 is the nozzle mixer; 12 is the left-right swing drive unit; 13 is the front-back swing drive unit; 14 is the rotary drive unit; 15 is the air compressor; 16 is the concrete delivery pump; and 17 is the accelerator pump. Detailed Implementation

[0082] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0083] Example 1

[0084] like Figure 1 As shown, a TBM intelligent shotcrete system based on vision servoing includes:

[0085] The vision acquisition unit is used to acquire image information of the shotcrete working surface in real time.

[0086] The motion execution unit is equipped with a shotcrete end effector to perform shotcrete spraying on the shotcrete working surface;

[0087] The feeding unit delivers various types of slurry to the shotcrete end effector.

[0088] The intelligent control unit analyzes the image information acquired by the vision acquisition unit, compares it with the target image information threshold, and controls the motion execution unit, the shotcrete end effector, and the material feeding execution unit to perform the corresponding actions.

[0089] The feeding execution unit is connected to the motion execution unit, and the feeding execution unit supplies slurry to the shotcrete end effector on the motion execution unit. The vision acquisition unit, motion execution unit, and feeding execution unit are all connected to the intelligent control unit. Both the vision acquisition unit and the shotcrete end effector are movably mounted on the rotary platform 4, and both can perform circular motion along the rotary platform 4. Figure 2 As shown, the rotary platform 4 is slidably mounted on the traveling platform 5, meaning the rotary platform 4 can move horizontally along the traveling platform 5. A motor reducer is installed on the rotary platform 4, and a rack is installed on the traveling platform 5. The gears on the motor reducer mesh with the rack to achieve linear movement of the rotary platform 4 along the traveling platform 5, thereby increasing the coverage area of ​​the shotcrete. Alternatively, a hydraulic cylinder can be installed between two rotary platforms 4, with a linear guide rail installed on the traveling platform 5 and a slider installed on the rotary platform 4. The hydraulic cylinder then enables the rotary platform 4 to move linearly along the traveling platform 5.

[0090] The vision acquisition unit integrates a multispectral industrial camera, including but not limited to RGB and infrared cameras. Since the image acquisition unit is exposed to the intense spraying environment for extended periods, an anti-adhesion coating is used to protect the camera. This anti-adhesion coating is applied directly to the glass during camera encapsulation and mounted in front of the camera lens. The motion execution unit is a work servo actuator that houses a multi-degree-of-freedom intelligent spraying robotic arm. The end effector of the work servo actuator is a spraying end effector, which performs forward, backward, oscillating, left-right, and brushing movements. The material supply unit includes multiple slurry conveying systems. Pressure sensors and flow sensors are installed on each slurry conveying system to enable real-time adjustment of the pressure and flow rate of each system under different spraying conditions. The intelligent control unit is equipped with a processor and a control module. The processor collects and sends various signals from various sensors, performs data processing, and converts them into motion control signals, which are then sent to the control module. The control module sends motion commands to each joint to achieve the corresponding motion response. At the same time, a core analysis algorithm is deployed. The image information collected by the vision acquisition unit is used as input. After the core analysis algorithm is executed, the data is converted into the motion of each joint of the motion execution unit and the pressure, flow rate, and other parameters of each slurry conveying system of the material supply execution unit as output, thereby realizing intelligent control of the shotcrete operation.

[0091] Example 2

[0092] like Figure 1As shown, a TBM intelligent shotcrete system based on visual servoing is disclosed. The visual acquisition unit includes a first visual acquisition unit 1 and a second visual acquisition unit 2. The shotcrete end effector is a nozzle assembly 3. The first visual acquisition unit 1 and the second visual acquisition unit 2 are respectively arranged on both sides of the nozzle assembly 3. This arrangement allows the first visual acquisition unit 1 on the right side to acquire image information when the nozzle assembly 3 is performing shotcrete operations on the left side; conversely, the second visual acquisition unit 2 on the left side acquires image information when the nozzle assembly 3 is performing shotcrete operations on the right side. Because the shotcrete operation is performed along the rotating platform 4 at a 270-degree angle, when shotcreting on the left side, the slurry and rebound will inevitably fall to the lower left first due to gravity. Therefore, if the second visual acquisition unit 2 on the left side is still working at this time, the probability of its camera being contaminated will greatly increase. Thus, the left camera is turned off and protected, while the first visual acquisition unit 1 on the right side is turned on. This greatly reduces the probability of camera damage. Similarly, when shotcreting on the right side, the left camera is turned on, and the right camera is turned off and protected.

[0093] Furthermore, the installation configuration of the first visual acquisition unit 1 and the second visual acquisition unit 2 can be adjusted according to the nozzle assembly 3. For example, if the nozzle assembly 3 is relatively long, the visual acquisition unit can also be installed at a relatively long length. The principle is that the visual acquisition unit and the nozzle assembly 3 should be on the same cross section as much as possible when the visual acquisition unit is working, so that the camera's field of view can fully cover the sprayed area.

[0094] To ensure that the acquired images are not affected by the tunnel's lighting environment, both the first visual acquisition unit 1 and the second visual acquisition unit 2 integrate supplementary lighting arrays to provide real-time supplementary lighting to the target working surface. The supplementary lighting arrays can be mounted on the nozzle assembly 3, rotating as the nozzle assembly 3 rotates, ensuring consistent lighting conditions at all camera positions; alternatively, the light source can be arranged on the rotating platform 4, with the principle of maintaining consistent lighting conditions for the visual acquisition units when acquiring images.

[0095] The other structures and implementation methods are the same as in Example 1.

[0096] Example 3

[0097] like Figure 1 As shown, a TBM intelligent shotcrete system based on visual servoing is described. Both the first visual acquisition unit 1 and the second visual acquisition unit 2 include protective chambers, which are cylindrical bodies used to protect the cameras 6 of the first and second visual acquisition units 1 and 2 when not in use. Figure 3 and Figure 4As shown, a camera 6 is installed inside the protective chamber, which is used to collect image information of the shotcrete operation. A first water flushing assembly 7 and an air flushing assembly 8 are fixedly installed on the outer circumference of the camera 6. These components form an air curtain during image acquisition by the camera 6, effectively blocking dust. The camera 6, the first water flushing assembly 7, and the air flushing assembly 8 are all mounted on a fixed base, which is connected to a telescopic hydraulic cylinder 10. The telescopic hydraulic cylinder 10 drives the fixed base to extend or retract, thereby extending or retracting the camera 6, the first water flushing assembly 7, and the air flushing assembly 8 into the protective chamber, protecting the camera 6 when not in use. The telescopic hydraulic cylinder 10 is located at one end of the protective chamber, with its cylinder body positioned outside the chamber. The protective chamber and telescopic cylinder 10 are fixedly mounted on the rotary gear ring of the rotary platform 4. A rotary drive unit 14 is fixed inside the rotary gear ring, which drives the rotary gear ring to rotate, thereby causing the first visual acquisition unit 1, the second visual acquisition unit 2, and the nozzle assembly 3 to move in a circular motion along the rotary platform. A second water flushing assembly 9 is installed on the front circumference of the protective chamber. When the camera 6 returns to the protective chamber, the second water flushing assembly 9 flushes and reduces dust on the front of the camera 6. The air curtain dust blocking + liquid flushing dust reduction arrangement in this application ensures that the visual acquisition units remain uncontaminated.

[0098] Furthermore, camera 6 can be a monocular camera, a binocular camera, or a combination of a camera and an infrared camera. A monocular camera can provide real-time images or videos as input for data processing; a combination of a camera and an infrared camera can perform fog-penetrating processing in very dusty environments, enhancing the quality of image acquisition in dusty environments; a binocular camera can calculate the lining thickness during shotcreting by utilizing the viewing angle difference between the two lenses based on the camera's principle; these cameras have different costs and are selected according to actual needs during the process.

[0099] Since the excavation diameter of the TBM equipment is the same as the tunnel diameter, the TBM equipment is fixed relative to the tunnel. The TBM shotcrete equipment is also fixed to the TBM equipment, therefore the entire TBM shotcrete equipment is fixed relative to the tunnel. The rotary platform 4 is installed on the TBM equipment, and its position is fixed relative to the tunnel. The nozzle assembly 3 is fixed on the rotary gear ring of the rotary platform 4, and it has circumferential rotational motion relative to the tunnel. Therefore, the position of the nozzle relative to the tunnel, i.e., the position of the shotcrete, can be determined by reading the angle of the encoder of the rotary drive unit of the rotary platform 4. The specific method is as follows: First, rotate the rotary drive unit 14 of the rotary platform 4 to the lowest point in a certain direction of the rotary gear ring, and mark the encoder position of the rotary drive unit at this point as 0; the rotation of the rotary drive unit 14 drives the nozzle assembly 3 to rotate, and the angle value of the encoder of the rotary drive unit is read. This angle value is the rotation angle of the rotary drive unit 14 relative to the tunnel axis, and it is also the rotation angle of the nozzle assembly 3 relative to the tunnel axis, thus determining the relative position of the shotcrete relative to the tunnel.

[0100] like Figure 3 As shown, the nozzle assembly 3 includes a nozzle mixer 11, which is connected to a left-right swing drive unit 12 and a front-back swing drive unit 13. The left-right swing drive unit 12 and the front-back swing drive unit 13 are connected in series. One method involves first connecting the front-back swing drive unit 13 to the structural components of the nozzle mixer 11 via a swing cylinder, controlling the front-back swing of the nozzle mixer 11 and the preceding nozzle assembly 3. Then, the left-right swing drive unit 12 is connected to the structural components where the front-back swing drive unit 13 is installed, enabling front-back swing and left-right swing of the overall front structure. The left-right swing drive unit 12 and the front-back swing drive unit 13 are fixed to the rotary gear ring of the rotary platform 4 via a fixed base. The nozzle mixer 11 is connected to a slurry conveying system, which delivers various slurries into the nozzle mixer 11. The slurry delivery system includes an air compressor 15, a concrete delivery pump 16, and an accelerator pump 17. Both the concrete delivery pump 16 and the accelerator pump 17 are connected to the air compressor 15. The air compressor 15 is connected to the nozzle mixer 11. The air compressor 15 provides the pressure for concrete slurry injection. The concrete delivery pump 16 serves as the concrete delivery actuator, and the accelerator pump 17 serves as the accelerator delivery actuator. The high-pressure air, concrete, and accelerator are simultaneously delivered to the nozzle and mixed before being sprayed onto the tunnel wall to complete the mixing and spraying action.

[0101] In the TBM intelligent shotcrete system of the present invention, such as Figure 5As shown, a first visual acquisition unit 1 and a second visual acquisition unit 2 are arranged on both sides of the shotcrete end actuator. During tunnel shotcrete support, the air compressor 15, concrete pump 16, and accelerator pump 17 of the slurry delivery system are first started to complete the pre-shotcrete preparation. After the shotcrete operation starts, the visual acquisition units begin to work. Simultaneously, to minimize contamination of the visual acquisition units by rebounding slurry, the control module reads the encoder angle of the rotary drive unit to determine the shotcrete position. That is, the position of the nozzle assembly 3 is automatically obtained using the angle measured by the encoder of the rotary drive unit. Taking a rotation angle of 270° as an example, 0-135° is defined as the left side, and 136-270° as the right side. When performing shotcrete operation in the left area, the first visual acquisition unit 1 on the right extends its protective chamber to a position convenient for observing the shotcrete status to acquire image information, while the second visual acquisition unit 2 on the left retracts into its protective chamber. The opposite is true when performing shotcrete operation in the right area. Meanwhile, due to the high dust levels in the shotcrete operation space, the camera 6's casing is coated with an anti-adhesion layer. During shotcrete operations, the air-washing component 8 on the visual acquisition unit currently acquiring images uses air jets to form an air curtain for protection. Simultaneously, at 1-minute intervals, the first water-washing component 7 washes the camera 6's casing with water to ensure the cleanliness of the front of the camera 6. When the visual acquisition unit, which is no longer acquiring images, returns to the protective chamber, the second water-washing component 9 on the protective chamber washes the cylinder rod of the telescopic cylinder 10 and the camera 6, ensuring the cleanliness of the cylinder rod and the channel. The acquired image information is transmitted to the intelligent control unit via video or image processing. The target area of ​​the shotcrete operation is marked in the image information, and feature information within this target area is extracted. Feature information may include grayscale, RGB image texture, etc., for example, using image grayscale values ​​for servo control. First, the intelligent control unit establishes an image grayscale threshold G1 when the shotcrete quality meets the standard. The real-time image grayscale value ΔG is compared with the image grayscale threshold G1 in real time. When ΔG≠G1, the residence time of the nozzle mixer 11, the nozzle angle, the real-time flow rate of the concrete delivery pump 16, the real-time flow rate of the accelerator pump 17, and the pressure of the air compressor 15 are dynamically adjusted until ΔG=G1. Then, the operation servo actuator of the shotcrete robotic arm is moved to the adjacent non-compliant area to repeat the above shotcrete steps and complete the shotcrete operation in the work area.

[0102] The other structures and implementation methods are the same as in Example 1.

[0103] Example 4

[0104] like Figure 1As shown, a TBM intelligent shotcrete system based on visual servoing is disclosed. Solenoid valves are installed at the front ends of the first water flushing component 7, the air flushing component 8, and the second water flushing component 9 to control the opening and closing states of the water and air flushing components. The solenoid valves are connected to a processor. Each of the first water flushing component 7, the air flushing component 8, and the second water flushing component 9 includes a nozzle. The nozzles of the first and second water flushing components 7 and 9 are first connected to the solenoid valves and then to the high-pressure water pump. The nozzle of the air flushing component 8 is first connected to the solenoid valves and then to the air compressor 15.

[0105] The specific principles and procedures for water and air flushing are as follows:

[0106] One visual acquisition unit is installed on each of the left and right sides of the nozzle assembly 3. Its control logic is as follows: when the nozzle assembly 3 sprays grout to the left along the tunnel axis, the first visual acquisition unit on the right is activated; when the nozzle sprays grout to the right circumferentially, the visual acquisition unit on the left is activated. The activation and deactivation logic of the visual acquisition units on both sides is consistent. Simultaneously, the installation and control logic of the water flushing assembly and the air flushing assembly 8 are consistent. The following explanation uses a single-sided visual acquisition unit as an example:

[0107] 1. When the vision acquisition unit is closed, the telescopic cylinder 10 drives the camera 6, the first water flushing component 7 and the air flushing component 8 to retract into the protective chamber. Because the vision acquisition unit is exposed to the sprayed slurry environment before retraction, slurry contamination is inevitable. Therefore, during the retraction process, the retraction action is interlocked with the second water flushing component 9. The retraction begins to spray water to flush and clean the vision device and the cylinder rod of the telescopic cylinder 10 until it is fully retracted.

[0108] 2. When the vision acquisition unit is activated, the telescopic cylinder 10 drives the vision component to extend out of the protective chamber. Because the vision acquisition unit may be contaminated by diffused dust before extension, the telescopic action is interlocked with the first water rinsing component 7 during the process. Water spraying begins as soon as the unit extends, rinsing the lens cover of the camera 6 until it is fully extended. After extension, the first water rinsing component 7 is closed, and the air rinsing component 8 is activated to wash away any remaining water droplets on the lens cover, ensuring the acquisition accuracy of the camera 6.

[0109] 3. During the shotcrete process, the air flushing component 8 is always open to form an air curtain for protection. At the same time, the first water flushing component 7 is opened at certain intervals to flush the lens and ensure its cleanliness during the process.

[0110] Therefore, according to the present invention, the tunnel surface features and the features of the shotcrete lining can be obtained in real time, and then the execution unit can be controlled in real time to execute the corresponding shotcrete action and shotcrete parameters, so as to achieve the purpose of intelligent shotcrete closed-loop control and better realize the precise control of shotcrete thickness and flatness.

[0111] Example 4

[0112] like Figure 5 As shown, a control method for a TBM intelligent shotcrete system based on visual servoing is described. After shotcreting starts, the control module reads the encoder angle of the rotary drive unit to determine the shotcrete position. If the shotcrete position is between 0-135°, that is, the nozzle assembly 3 is located on the left side of the rotary platform 4, the first visual acquisition unit 1 is pushed by the telescopic cylinder 10 to extend the camera 6 in the protective chamber to acquire shotcrete images and tunnel wall images. When extended, the second water flushing assembly 9 is opened to flush the surface of the telescopic cylinder 10, ensuring its cleanliness. At the same time, during image acquisition, the air flushing assembly 8 is always open to form an air curtain, protecting the camera 6 from contamination by rebounding concrete. Simultaneously, the first water flushing assembly 7 is opened at intervals to clean the concrete on the camera 6. When the first visual acquisition unit 1 extends, the second visual acquisition unit 2 retracts. During the retraction process, the first water flushing assembly 7, the air flushing assembly 8, and the second water flushing assembly 9 of the visual acquisition unit are opened simultaneously to ensure the cleanliness of the telescopic cylinder and the camera. Simultaneously, when the spraying position is between 136-270°, i.e., the nozzle assembly 3 is located on the right side of the rotary platform 4, the motion logic of the first and second vision acquisition units is opposite. The acquired image information is transmitted to the intelligent control unit via video or image, marking the target range of the spraying operation in the image information, and extracting feature information within that target range. Feature information can include grayscale, RGB image texture, etc. For example, servo control can be performed using image grayscale values. First, an image grayscale threshold G1 is established when the spraying quality meets the standard. The real-time image grayscale value ΔG is compared with G1 in real time. When ΔG≠G1, the nozzle dwell time, nozzle angle, real-time flow rate of the concrete delivery pump, real-time flow rate of the accelerator pump, and air compressor pressure are dynamically adjusted according to the following mapping relationship: |ΔG- G1|=|T、α、β、K、X、S|, where T is the shotcrete dwell time, α is the left-right swing angle of the nozzle, β is the front-back swing angle of the nozzle, K is the real-time flow rate of the concrete delivery pump, X is the real-time flow rate of the accelerator pump, and S is the pressure of the air compressor. When ΔG=G1, move the shotcrete robotic arm to the adjacent qualified area and repeat the above shotcrete steps to complete the shotcrete operation in the work area.

[0113] The grayscale threshold G1 for achieving the required shotcrete quality needs to be determined by considering both the shotcrete quality standard and the on-site environment. First, sampling training is conducted: shotcrete images are sampled at the tunnel construction site. For manual shotcreting on-site, no parameter intervention is performed; only image sampling of the shotcrete results is performed. For locations manually considered "satisfied" by the operator, samples of the tunnel surface after shotcreting are collected using a visual acquisition unit consistent with the on-site conditions and under consistent lighting. Areas deemed to have acceptable shotcrete quality are selected in the image annotation tool and marked as "acceptable areas." The number of physical samples should be as large as possible, with no fewer than 500 images.

[0114] Then, standard image grayscale feature extraction is performed: the RGB images in the sample are converted to grayscale values ​​using a weighted average method, with the following formula: The value range is 0-255;

[0115] Where R, G, and B are the pixel values ​​of the red, green, and blue channels of the image, respectively. , , These are all weighting coefficients, which can be customized according to the tunnel environment. In the application of this invention in a certain tunnel, the corresponding values ​​are 0.2126, 0.7152, and 0.0722. Simultaneously, before assigning values, a filtering method is used to remove noise interference caused by dust and slurry splashes in the shotcrete image, avoiding the influence of invalid grayscale values.

[0116] The shotcrete images of the "qualified areas" collected above were statistically analyzed, and the arithmetic mean of the grayscale values ​​of all the shotcrete images of the "qualified areas" was recorded as the grayscale mean. .

[0117] Determine the initial image grayscale threshold:

[0118] An image of a shotcrete process (including qualified and unqualified shotcrete areas) is acquired. Areas with a grayscale value ≥ k (range 0-255) are defined as "qualified shotcrete areas," and their pixel count accounts for a certain percentage of the total image. The average grayscale value of these pixels is The area with a grayscale value < k (range 0-255) is designated as the "unqualified sprayed mortar area," and its pixel count accounts for a certain percentage of the entire image. The average grayscale value of these pixels is ;k represents the specific grayscale threshold, which ranges from 0 to 255 and is the industry standard for grayscale values ​​in digital images.

[0119] Using the formula for calculating inter-class variance, an initial optimal image threshold is set. for:

[0120] =

[0121] Iterate through the values ​​of k from 0 to 255 and calculate the inter-class variance for each k value. Find the variance between classes The maximum value of k is set as the initial optimal image threshold. .

[0122] Optimal image threshold based on the actual tunnel environment Make corrections and dynamically adapt and adjust.

[0123] Identify influencing factors at the construction site, including tunnel lighting intensity and dust concentration. Quantify these factors using sensors integrated into the tunnel boring machine: Light Intensity: Obtain the current ambient brightness through the lighting sensor integrated into the vision acquisition unit. (unit: Dust concentration: The mass concentration of dust in the air is obtained through dust sensors inside the tunnel. (unit: ).

[0124] Based on the following compensation formula, the initial optimal image threshold is... The image grayscale threshold for achieving the final shotcrete quality standard is dynamically determined through correction. for:

[0125]

[0126] , To determine the optimal image threshold at the initial calibration The tunnel lighting and dust environment were recorded in real time. , The corresponding compensation coefficients are set according to the tunnel site environment. In the application of this invention in a certain tunnel, the corresponding values ​​are set to -0.02 and 0.04.

[0127] Based on real-time image information collected during the shotcreting process, the real-time grayscale value of the shotcreting area is calculated. Image grayscale threshold Based on the deviation characteristics between them, the improved neural network guides each actuator of the intelligent shotcrete system to make corresponding time and pose adjustments.

[0128] Calculation of grayscale deviation characteristics:

[0129] Absolute deviation: ;

[0130] Relative deviation: , and define It is a slight deviation. The deviation is moderate. There is a slight height deviation;

[0131] Deviation change rate: ,in, The sampling frequency; This represents the absolute grayscale deviation of the sprayed area at the current time, i.e., time t. This grayscale deviation is obtained through image analysis and is used to measure the difference in grayscale characteristics between the sprayed area and the standard sprayed area, such as the absolute deviation between the grayscale of the actual sprayed area and the grayscale of the qualified sprayed area. This represents the absolute grayscale deviation value of the shotcrete area at the previous time, i.e., time t-1, and It's the same type of grayscale deviation, just one sampling interval earlier.

[0132] Gray variance: , The number of pixels in the target area. , This represents the grayscale value of the i-th pixel within the target area. When calculating the grayscale variance, it is necessary to consider the grayscale value of each pixel within the target area and the average grayscale value of that area. The difference. Based on construction experience: when the grayscale variance... Uneven grayscale distribution in the defined area likely indicates insufficient shotcrete thickness; when the grayscale variance... The definition indicates that the area with uniform grayscale distribution is likely insufficient in terms of shotcrete thickness. The target area can be manually selected based on experience on the acquired shotcrete area image; alternatively, image recognition algorithms can be used to automatically identify areas with acceptable shotcrete thickness as the target area.

[0133] The improved neural network achieves the mapping from image features to the parameters of various actuators in the jetting process through an "input-feature extraction-output" signal transmission process. The improved neural network structure can be represented as follows:

[0134] Input layer: consists of 4 neurons, corresponding to 4 gray-level deviation features:

[0135]

[0136] (Absolute deviation) (Relative deviation) (Rate of change of deviation) (Gray variance).

[0137] Hidden Layer I: Each neuron performs a weighted sum of the input parameters, and then... Activation functions filter out invalid information;

[0138] Weighted summation: The first hidden layer I one neuron The input is:

[0139]

[0140] It is the first of the input layer The parameters are passed to the hidden layer I. The weights of each neuron, such as Indicates input parameters For the The degree of influence on each neuron; This is a bias term, similar to a constant term in a linear equation, used to adjust the sensitivity threshold of a neuron. Weights Random initialization occurs during training. Before training begins, weights are assigned initial values ​​according to a certain distribution (such as Gaussian or uniform distribution). The purpose of this is to break symmetry, allowing neurons to have different responses in the early stages of training, thus enabling more effective feature learning. Bias term. The initial setting is 0. The purpose of the bias term is to adjust the sensitivity threshold of neurons. Initializing it to 0 allows the network to learn patterns in the data in a relatively "neutral" state at the beginning of training. Subsequently, the value of the bias term is adjusted according to the error during the training process (such as the backpropagation algorithm) to make it more suitable for responding to the input data.

[0141] Activation function for

[0142]

[0143] If input If the input is 0, the output is 0, indicating that the neuron "does not respond" to the current input; if the input is 0, the output is 0. Then output This indicates that the feature is activated and transmitted.

[0144] Hidden Layer II: Hidden Layer II outputs 16 low-order features from Hidden Layer I. arrive A second combination is performed to generate higher-order features.

[0145] Weighted summation: The first hidden layer II one neuron The input is:

[0146]

[0147] It is the first hidden layer I The first neuron to hidden layer II The weights of each neuron. This is a bias term, similar to a constant term in a linear equation, used to adjust the sensitivity threshold of a neuron. Weights Initializes randomly during training. Bias term. Initialize to 0.

[0148] Similarly, Activation functions filter out invalid information;

[0149]

[0150] If input If the input is 0, the output is 0, indicating that the neuron "does not respond" to the current input; if the input is 0, the output is 0. Then output This indicates that the feature is activated and passed to the next hidden layer.

[0151] Output layer: Hidden layer II outputs 12 higher-order features ( The output parameters are directly correlated, and the execution parameters are obtained by summing the last weighted sum and applying it to the activation function.

[0152] Weighted summation: Output layer The input of each neuron ( ):

[0153]

[0154] Activation function for

[0155]

[0156]

[0157] in, Residence time of the nozzle , The left and right swing angle of the nozzle , The angle of the nozzle's back-and-forth swing , Real-time flow rate of shotcrete pump , Real-time flow rate of accelerator pump , air compressor pressure . , These are the maximum and minimum thresholds for the corresponding parameters, such as dwell time. The duration is 0.5-3.0 seconds, depending on the on-site construction conditions. The intermediate parameters used in the calculation are the actual output parameters. The previous predicted value. The number of neurons in the output layer is determined based on the output parameters. In this invention, 6 parameters are required for output, so the output layer has 6 neurons. The number of neurons in hidden layer I and hidden layer II needs to be trained and evaluated. For example, first set 16 neurons in hidden layer I and 12 neurons in hidden layer II, train them, and determine whether the parameters of the output layer meet the performance indicators. If they are not suitable, they need to be adjusted.

[0158] After establishing the above model, the improved neural network was trained using on-site collected data. Simultaneously, the weights of hidden layer I, hidden layer II, and the output layer were redistributed, and a new round of training was conducted, utilizing the loss function. To conduct an evaluation, These are the actual shotcrete parameters. These are predicted values.

[0159] When the loss function The model training is complete at 5%.

[0160] In the subsequent actual spraying, only image acquisition and automatic input of the parameters of the input layer are required to output 6 execution parameters and realize closed-loop servo control.

[0161] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A real-time monitoring and control system for TBM shotcrete quality based on vision servoing, characterized in that, include: The vision acquisition unit is used to acquire image information of the shotcrete working surface in real time. The motion execution unit is equipped with a shotcrete end effector to perform shotcrete spraying on the shotcrete working surface; The feeding unit delivers various types of slurry to the shotcrete end effector. The intelligent control unit analyzes the image information acquired by the vision acquisition unit, compares it with the target image information threshold, and controls the shotcrete end effector and the material feeding execution unit to perform corresponding actions. The material feeding execution unit is connected to the shotcrete end effector, and the vision acquisition unit, motion execution unit, and material feeding execution unit are all connected to the intelligent control unit.

2. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 1, characterized in that, The visual acquisition unit and the shotcrete end effector are both movably mounted on the rotary platform (4), and the rotary platform (4) is slidably mounted on the traveling platform (5); The vision acquisition unit integrates a multispectral industrial camera, which includes, but is not limited to, RGB cameras and infrared cameras. An anti-adhesion coating is applied to the glass in front of the industrial camera. The motion execution unit is a work servo execution mechanism with a multi-degree-of-freedom intelligent shotcrete robotic arm. The end of the work servo execution mechanism is equipped with a shotcrete end effector. The work servo execution mechanism realizes the forward, backward, back-and-forth swing, left-and-right swing, and brushing motion of the shotcrete end effector. The feeding execution unit includes multiple slurry conveying systems, and pressure sensors and flow sensors are installed on each slurry conveying system; The intelligent control unit includes a processor and a control module. The vision acquisition unit, pressure sensor, and flow sensor are all connected to the processor. The processor is connected to the control module, and the control module is connected to the work servo actuator, the shotcrete end effector, and the slurry conveying system, respectively.

3. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 1 or 2, characterized in that, The visual acquisition unit includes a first visual acquisition unit (1) and a second visual acquisition unit (2). The shotcrete end effector is a nozzle assembly (3). The first visual acquisition unit (1) and the second visual acquisition unit (2) are respectively set on both sides of the nozzle assembly (3). When the nozzle assembly (3) performs shotcrete operation in the left area, the first visual acquisition unit (1) on the right side performs image information acquisition; when the nozzle assembly (3) performs shotcrete operation in the right area, the second visual acquisition unit (2) on the left side performs image information acquisition.

4. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 3, characterized in that, The first visual acquisition unit (1) and the second visual acquisition unit (2) extend out and are on the same cross section as the nozzle assembly (3) in the working state; The nozzle assembly (3) or the rotating platform (4) is provided with a supplementary light array consisting of a first visual acquisition unit (1) and a second visual acquisition unit (2).

5. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 4, characterized in that, The first visual acquisition unit (1) and the second visual acquisition unit (2) both include a protective chamber. A camera (6) is provided inside the protective chamber. A first water rinsing assembly (7) and an air rinsing assembly (8) are fixedly installed on the outer circumference of the camera (6). The camera (6), the first water rinsing assembly (7) and the air rinsing assembly (8) are all installed on a fixed base. The fixed base is connected to a telescopic cylinder (10). The telescopic cylinder (10) is located at one end of the protective chamber, and the cylinder body of the telescopic cylinder (10) is located on the outside of the protective chamber. A second water flushing assembly (9) is installed on the circumference of the front part of the protective chamber.

6. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 5, characterized in that, The protective chamber and the cylinder body of the telescopic cylinder (10) are fixedly installed on the rotary gear ring of the rotary platform (4). A rotary drive unit (14) is fixed inside the rotary gear ring, and the rotary drive unit (14) drives the rotary gear ring to rotate. The camera (6) is a monocular camera, a binocular camera, or a combination of a camera and an infrared camera; The nozzle assembly (3) is fixed on the rotary gear ring of the rotary platform (4). The position of the nozzle assembly (3) relative to the tunnel is determined by reading the angle of the encoder of the rotary drive unit of the rotary platform (4).

7. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to any one of claims 4-6, characterized in that, The nozzle assembly (3) includes a nozzle mixer (11), which is connected to the left and right swing drive unit (12) and the front and back swing drive unit (13) respectively. The left and right swing drive unit (12) and the front and back swing drive unit (13) are connected in series. The left and right swing drive unit (12) and the front and back swing drive unit (13) are both fixed on the rotary gear ring of the rotary platform (4) by a fixed base. The nozzle mixer (11) is connected to the slurry conveying system.

8. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 7, characterized in that, The slurry conveying system includes an air compressor (15), a concrete conveying pump (16) and a quick-setting agent pump (17). The concrete conveying pump (16) and the quick-setting agent pump (17) are both connected to the air compressor (15), and the air compressor (15) is connected to the nozzle mixer (11).

9. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 5 or 6, characterized in that, The front ends of the first water flushing assembly (7), the air flushing assembly (8), and the second water flushing assembly (9) are all equipped with solenoid valves, which are connected to the control module. The nozzles of the first water flushing assembly (7) and the second water flushing assembly (9) are connected to the high-pressure water pump through solenoid valves, and the nozzles of the air flushing assembly (8) are connected to the air compressor (15) through solenoid valves.

10. The TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 9, characterized in that, During the retraction of the visual acquisition unit, the retraction action is interlocked with the second water flushing component (9) until it retracts to the correct position; when the visual acquisition unit extends, the extension and retraction action is interlocked with the first water flushing component (7) until it extends to the correct position; the first water flushing component (7) is closed, and the air flushing component (8) is opened to flush the water droplets remaining on the lens cover of the camera (6); during the spraying operation, the air flushing component (8) is normally open to form an air curtain protection, and at the same time, the first water flushing component (8) is opened at a set time interval for flushing.

11. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to any one of claims 1-10, characterized in that, When performing shotcrete support in the tunnel, the air compressor (15), concrete pump (16), and quick-setting agent pump (17) of the grout delivery system are started first to complete the preparations before shotcreting. After the shotcreting operation is started, the first vision acquisition unit (1) and the second vision acquisition unit (2) start working and automatically obtain the position of the nozzle assembly (3) by measuring the angle of the encoder of the rotary drive unit. When performing shotcreting operation in the left area, the camera of the first vision acquisition unit (1) on the right extends out of the protective chamber to a position that is convenient for observing the shotcreting status, and the second vision acquisition unit (2) on the left retracts into the protective chamber. The opposite is true when performing shotcreting operation in the right area. During extension and retraction, the second water flushing is performed. The component (9) opens to rinse the camera (6) with water; during the shotcrete operation, the air flushing component (8) on the visual acquisition unit that acquires images sprays air to form an air curtain for protection. At the same time, after reaching the set interval, the first water flushing component (7) is opened to rinse the outer cover of the camera (6) with water. During the process of the visual acquisition unit retracting into the protective chamber, the second water flushing component (9) arranged on the protective chamber rinses the cylinder rod of the telescopic cylinder (10) and the camera (6); the image information acquired by the visual acquisition unit is transmitted to the intelligent control unit through video or image, the target range of the shotcrete operation in the image information is marked, and the feature information within the target range is extracted. Use feature information to determine whether the shotcrete quality meets the standards.

12. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 11, characterized in that, The intelligent control unit establishes an image grayscale threshold G1 based on the grayscale of the feature information when the shotcrete quality meets the standard. The image grayscale threshold G1 is compared with the real-time image grayscale value ΔG in real time. When ΔG≠G1, the residence time of the nozzle mixer, the nozzle angle, the real-time flow rate of the concrete delivery pump, the real-time flow rate of the accelerator pump, and the pressure of the air compressor are dynamically adjusted until ΔG=G1. Then, the operation servo actuator of the shotcrete robotic arm is moved to the adjacent non-compliant area to repeat the above shotcrete steps and complete the shotcrete operation in the work area.

13. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 12, characterized in that, Dynamic adjustments are made based on the mapping relationship |ΔG - G1| = |T, α, β, K, X, S|, where T is the shotcrete dwell time, α is the left-right swing angle of the nozzle, β is the front-back swing angle of the nozzle, K is the real-time flow rate of the concrete delivery pump, X is the real-time flow rate of the accelerator pump, and S is the pressure of the air compressor.

14. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 11 or 12, characterized in that, The image grayscale threshold G1 for achieving the required shotcrete quality is determined by combining the shotcrete quality standard and the on-site environment. The implementation method is as follows: Image sample collection: For locations deemed "qualified" by humans, use the same visual acquisition unit as the site and under the same lighting conditions to collect samples of the tunnel surface after spraying, and select the areas with qualified spraying quality in the image annotation tool and mark them as "qualified areas"; collect no less than 500 image samples. Image grayscale feature extraction: Convert the RGB images in the image samples into grayscale values ​​using a weighted average method; For each image sample from the acquired "qualified region", the arithmetic mean of the grayscale values ​​of all the "qualified region" image samples is calculated and recorded as the grayscale mean. ; The initial optimal image threshold is set using the formula for calculating inter-class variance; The optimal image threshold is dynamically corrected based on the actual tunnel environment to obtain the image grayscale threshold G1 that meets the shotcrete quality standard.

15. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 14, characterized in that, Before extracting grayscale features from the image, a filtering method is used to remove noise interference caused by dust and slurry splashing in the spraying image; The method for determining the initial optimal image threshold is as follows: Image samples from a specific spraying process are collected, and regions with grayscale values ​​≥ k are designated as "qualified regions." The proportion of pixels in the "qualified regions" to the total image size is... The average grayscale value of the pixels in the "qualified area" is The region with a grayscale value < k is designated as the "unacceptable region," and its pixel count accounts for a certain percentage of the total image. The average grayscale value of the pixel is ;k represents the grayscale threshold, ranging from 0 to 255; Using the formula for calculating between-class variance, the between-class variance is calculated as follows: = ； Iterate through the grayscale value thresholds k from 0 to 255, and calculate the inter-class variance for each grayscale value threshold k. Find the variance between classes The value of the grayscale threshold k at the maximum value is the initial optimal image threshold. .

16. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 15, characterized in that, The method for dynamically correcting the optimal image threshold is as follows: identify the light intensity and dust concentration at the construction site, and obtain the current ambient brightness through a light sensor. The mass concentration of dust in the air is obtained through dust sensors inside the tunnel. ; for the initial optimal image threshold The image grayscale threshold for achieving the final shotcrete quality standard is dynamically determined through correction. for: , The optimal image thresholds at the initial calibration are respectively The light intensity and dust concentration were recorded at the time. , These are the compensation coefficients for illumination intensity and dust concentration, respectively.

17. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 12 or 13, characterized in that, Based on real-time image information acquired during the shotcreting process, the real-time grayscale value of the image within the shotcreting area is calculated. Image grayscale threshold The grayscale deviation characteristics between them are used to guide the various actuators of the intelligent shotcrete system to make corresponding time and pose adjustments based on the neural network.

18. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 17, characterized in that, The calculation method for the grayscale deviation feature is as follows: Calculate the absolute deviation: ; Calculate the relative deviation: And relative deviation It is a slight deviation. The deviation is moderate. There is a slight height deviation; Deviation change rate: ,in, The sampling frequency; This represents the absolute grayscale deviation value related to the shotcrete area at time t. This represents the absolute grayscale deviation value related to the shotcrete area at time t-1; Gray variance: , This represents the number of pixels in the sprayed area. , This represents the grayscale value of the i-th pixel within the sprayed area; Based on construction experience: when the grayscale variance Uneven grayscale distribution in the shotcrete area indicates insufficient shotcrete thickness; when the grayscale variance... This indicates that the gray distribution in the sprayed area is uniform, indicating that the sprayed thickness is sufficient.

19. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 18, characterized in that, The neural network achieves the mapping from image feature information to the parameters of various actuators in the jetting process through an "input-feature extraction-output" signal transmission process. The neural network includes an input layer, hidden layer I, hidden layer II, and an output layer connected in sequence. The input layer includes four neurons, corresponding to four gray-scale deviation features: ;in, , , , ; Each neuron in hidden layer I performs a weighted sum of the input parameters, and then... Activation functions filter out invalid information; Hidden layer II performs a second weighted summation on the 16 low-order features output by hidden layer I to generate high-order features; Output layer: Hidden layer II outputs 12 higher-order features, which are directly related to the output parameters and are then processed by a final weighted summation and activation function. Obtain the execution parameters.

20. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 19, characterized in that, The weighted summation method for hidden layer I is as follows: the first... The input to each neuron is: It is the first of the input layer Parameters To the first hidden layer I The weights of each neuron, ; It is a bias term; The activation function of the hidden layer I for ; The weighted summation method for hidden layer II is as follows: the first... The input to each neuron is: It is the first hidden layer I The first neuron to hidden layer II The weights of each neuron, ; It is a bias term; Hidden Layer II The activation function filters out invalid information as follows: ; The weighted summation of the output layer is: the first... Input to each neuron: in, It is the second hidden layer II The first neuron to the output layer The weights of each neuron, ; For bias terms; The method for obtaining the execution parameters is as follows: in, For activation function, These are the actual shotcrete parameters, and The residence time of the nozzle , The left and right swing angle of the nozzle assembly , The forward and backward swing angle of the nozzle assembly , Real-time flow rate of shotcrete delivery pump , Real-time flow rate of accelerator pump , air compressor pressure ; , These are the maximum and minimum thresholds for the corresponding parameters. These represent intermediate parameters in the calculation, which are the actual output parameters. Previous forecast values.

21. The control method for the TBM shotcrete quality real-time monitoring and control system based on vision servoing according to claim 20, characterized in that, The weight Weight and weight The bias term is randomly initialized during training. Bias terms and bias terms Initialize to 0; The neural network model was trained using data collected on-site. Simultaneously, the weights of hidden layer I, hidden layer II, and the output layer were redistributed, and a new round of training was conducted, utilizing the loss function. To conduct an evaluation; When the loss function The model training is complete at 5%.

Images