TBM rock slag grade identification system based on synthetic data set and deep learning

Abstract

The invention discloses a TBM rock slag grade identification system and method based on a synthetic data set and deep learning. The TBM rock slag grade identification system comprises the synthetic data set and a rock slag grade identification system. According to the method, the synthetic data set is generated by using the Unity virtual engine, the problem of insufficient rock slag image samples is effectively solved, the data annotation cost is reduced, meanwhile, the data set is optimized through reinforcement learning, the similarity between the synthetic data set and a real data set is improved, and better data support is provided for a rock slag grade identification model; according to the rock slag grade classification algorithm based on deep learning design, an appropriate convolutional neural network is selected as a basic network through comprehensive evaluation, efficient and accurate rock slag grade identification can be realized, and the identification precision and efficiency are greatly improved; an automatic rock slag grade identification means is provided, tunneling parameters can be adjusted in time, construction efficiency is improved, tool abrasion is reduced, construction risks are reduced, and the method has important engineering application value.

Description

Technical Field

[0001] The present invention relates to a TBM rock slag grade recognition system, in particular to a TBM rock slag grade recognition system and method based on a synthetic data set and deep learning, belonging to the technical field of tunnel construction. Background Art

[0002] The full-face hard rock tunnel boring machine (TBM) plays an important role in tunnel construction, and its construction condition can be reflected by the degree of fragmentation of the rock slag during the tunneling process. Rock slag grade recognition is of great significance for monitoring the tunneling state and adjusting the tunneling parameters. For example, too many large particles of rock slag imply that the tunneling speed is too fast, which will exacerbate tool wear and even cause the cutting tool head to jam; too many small particles of rock slag indicate low tunneling efficiency. At present, the artificial observation method is mostly used at the TBM construction site to monitor the rock slag condition, relying on manual continuous monitoring of the rock slag on the conveyor belt. However, for the construction of hard rock geological tunnels, the construction line is long, the propulsion speed is slow and it needs to be constructed day and night. This traditional method is boring and time-consuming, which is likely to make the monitoring personnel fatigued, resulting in information omission and potential risks. At the same time, the training of artificial neural networks requires a large amount of labeled data, and the data set annotation work is time-consuming and laborious, and the manual annotation error is large, which seriously restricts the practical application of the rock slag grade recognition model.

[0003] Existing rock slag grade recognition methods, such as a method for recognizing the size grade of TBM rock slag based on a convolutional neural network disclosed in Publication No. CN110245695A, include: collecting the original images of TBM rock slag and constructing a rock slag image sample library in a single scenario; using a convolutional neural network to iteratively train the sample images, optimizing the network parameters, and obtaining an optimized convolutional neural network model; inputting the test set images into the optimized convolutional neural network model, the size grade of the initially crushed rock slag can be quickly and accurately evaluated through the images, and the TBM construction efficiency and fault diagnosis level are effectively improved. However, the existing recognition methods are based on the rock slag image sample library in a single scenario for image training and activities. There are many problems with methods such as thresholding and watershed segmentation. Directly using the watershed algorithm will cause serious over-segmentation, and the distinguishability of the rock slag particle size distribution is not strong. Although some researchers have proposed an optimized convolutional neural network model for rock slag images, the image samples used are relatively single, there is a problem of insufficient rock slag image samples, which cannot provide better data support for the rock slag grade recognition model, and processing multiple groups of graphic data in the sample library greatly increases the data annotation cost. Again, existing rock slag grade recognition methods, such as thresholding and watershed segmentation, have many problems. Directly using the watershed algorithm will cause serious over-segmentation, and the distinguishability of the rock slag particle size distribution is not strong. Although some researchers have proposed mapping the particle size statistical histogram of rock slag images to the screening analysis results, or using texture features combined with machine learning methods to extract the particle size distribution, a large number of experiments are still required to verify the effectiveness. In terms of synthetic datasets, although synthetic datasets based on virtual engines have applications in many fields, there is less research in the field of TBM rock slag grade recognition, and there is a domain gap between the synthetic data and the real data, reducing the generalization ability of the model on the real dataset. Summary of the Invention

[0004] The purpose of the present invention is to provide a TBM rock slag grade recognition system and method based on a synthetic dataset and deep learning to solve at least one of the above technical problems, solve the problems of low efficiency, poor accuracy of manual detection, and difficult data annotation in traditional rock slag grade recognition methods, and achieve efficient and accurate rock slag grade recognition, providing a reliable decision-making basis for TBM construction.

[0005] The present invention achieves the above purpose through the following technical solutions: A TBM rock slag grade recognition system based on a synthetic dataset and deep learning includes a synthetic dataset and a rock slag grade recognition system. The synthetic dataset includes:

[0006] Building a virtual scene: Choose to use Unity to build a virtual scene; collect 3D models of rock slag; build a conveyor belt model; use C# scripts to implement conveyor belt control; build a virtual camera; set adjustable parameters for the scene; export photos taken by the virtual camera; read the rendering cache of Unity to automatically generate labels for the dataset; export the synthetic dataset;

[0007] Optimizing the virtual scene through reinforcement learning: Obtain a real dataset for rock slag grade recognition; analyze project requirements and select a suitable reinforcement learning algorithm; use image similarity evaluation based on the hash algorithm as the basis for the reward function; formulate the reward function of reinforcement learning to provide timely feedback for the decision-making process; establish a neural network model according to the value function and the reward function; optimize the parameters of the virtual scene through the neural network model; obtain the optimal virtual scene parameters;

[0008] The rock slag grade recognition system includes: dividing the training set and the test set based on the synthetic dataset; comparing the algorithm efficiency and accuracy of different convolutional neural network models and selecting a suitable convolutional neural network model; training the convolutional neural network model of deep learning; optimizing the parameters of the rock slag grade recognition model; solidifying the optimal model parameters; evaluating the grade recognition effect of the model on real rock slag pictures.

[0009] As a further solution of the present invention: When building the virtual scene, use the python interface to give the initial parameters of the Unity virtual environment. Unity generates the virtual environment and takes pictures through the virtual camera to obtain a synthetic dataset, and compares the synthetic dataset with the real dataset for differences.

[0010] As a further solution of the present invention: The optimization of the virtual scene is based on the image similarity calculation program based on the hash algorithm as the basis for the reward function. The reward value is given according to the image similarity difference to update the value function; the agent takes the most valuable behavior according to the value function, updates the virtual environment parameters, repeats the training until the set number of times or the image similarity reaches the standard, and solidifies the value function; finally, adjusts the virtual scene parameters according to the solidified value function to obtain a synthetic dataset with labels similar to the real dataset.

[0011] As a further solution of the present invention: When the rock slag grade recognition system recognizes real rock slag pictures, the preprocessing of the real rock slag pictures includes: cropping the original rock slag image to remove unnecessary redundant backgrounds and reduce interference from irrelevant factors; then performing detail enhancement filtering to make the image details more obvious, while increasing the image contrast and saturation, enhancing the contrast of each part of the picture, and making the rock slag contour clearer;

[0012] The rock slag grade recognition system uses a synthetic dataset to test the accuracy and operation time of each network, comprehensively evaluates the image classification accuracy and algorithm efficiency, and selects a suitable network as the basic network for feature extraction. The selected basic networks include, but are not limited to: LeNet-5, AlexNet, VGG-16, Inception-v3 as candidate networks;

[0013] The training and optimization of the model by the rock slag grade recognition system specifically include: using the synthetic dataset as the input and the classification grades of large, medium, and small rock slags as the output, constructing a rock slag grade recognition model based on the selected basic network, and optimizing the model parameters by repeatedly training the convolutional neural network model, and finally solidifying the optimal model parameters;

[0014] The testing and evaluation of the model by the rock slag grade recognition system specifically include: using real rock slag images to test the trained model, and comprehensively evaluating the model performance using multiple indicators such as accuracy, precision, recall rate, and F value.

[0015] A method for recognizing the grade of TBM rock slag based on a synthetic dataset and deep learning, comprising the following steps:

[0016] S1. Generation of synthetic dataset

[0017] S11. In the Unity3D environment, collect a large number of 3D rock slag models and encapsulate them into prefabs, and attach rigidbody components; create a platform and a baffle, attach rigidbody components to them respectively and set them as static, and set up a camera and a light source directly above the platform;

[0018] S12. Set enough generation points on a horizontal plane slightly higher than the platform, and use a C# script to implement that the prefab of the rock slag model randomly selects a generation point for instantiation every 0.1s and is destroyed after n*0.1s, where n is the number of rock slag particles on the test bench; perform scaling of 0.8-1.2 times on the replicated body during instantiation, name them in sequence and endow them with rotation characteristics to simulate the actual slag discharge situation of the TBM;

[0019] S13. Use the RenderTexture library to write a script and attach it to the camera, create a folder and name it in sequence every n*m*0.1s, where m is the number of photos in the folder; render and save the pictures to the corresponding folder every n*0.1s after n*0.1s of running the virtual scene; use the EPPlus plugin to write a script and attach it to the camera, and export the position coordinates, length, width and other related parameters of the rock slag to the excel folder every n*0.1s after n*0.1s of running the virtual scene for convenient subsequent call;

[0020] S14. The Python interface uses socket communication to transmit initial parameters to the Unity virtual environment and start the reinforcement learning training. Unity generates a virtual environment and renders images through a virtual camera to obtain a synthetic dataset. The OpenCV library is used to read the images in the synthetic dataset and the real dataset, and the hash algorithm is used to calculate the image hash value. The hash value differences between the synthetic images and the real images are compared, the minimum difference value is selected and averaged to obtain the average difference value that quantifies the differences between the two datasets.

[0021] S15. Determine the reward value according to the image similarity calculation program based on the hash algorithm and update the value function. The agent selects actions according to the value function to update the virtual environment parameters and repeats the training process. When the set number of training times is reached or the image similarity meets the standard, stop the training and solidify the value function.

[0022] S16. Based on the solidified value function, use the Python interface again to give the initial parameters of the Unity virtual environment, select the action with the maximum value from the Q-table to adjust the virtual scene parameters, generate an optimized synthetic dataset, and add labels to each image to form a labeled dataset.

[0023] S2. Steps for identifying the slag grade

[0024] S21. After obtaining the slag image, first perform image preprocessing. Use an image editing tool or programming library to crop the original image and remove the background part irrelevant to the slag in the image. Then apply algorithms such as Gaussian filtering and Laplacian filtering for detail enhancement filtering to improve the image details. Enhance the contrast of each part of the image by adjusting the brightness, contrast, and saturation parameters of the image.

[0025] S22. Input the preprocessed image into candidate convolutional neural network models such as LeNet-5, AlexNet, VGG-16, and Inception-v3, and use the synthetic dataset to test the accuracy and operation time of each model. Record the classification accuracy of each model for the images in the synthetic dataset and the time required for the model to run. After comprehensive evaluation, select the model with high classification accuracy and fast operation efficiency as the basic network for slag grade identification.

[0026] S23. Build a slag grade identification model with the selected basic network as the framework. Use the images in the synthetic dataset as the input and label them according to the three slag grades of large, medium, and small as the output of the model. Use a deep learning training framework to train the model. During the training process, continuously adjust the weights and parameters of the model, and minimize the loss function through the backpropagation algorithm to enable the model to gradually learn the mapping relationship between the slag image features and the grades. After multiple rounds of training, solidify the optimal model parameters to obtain a maturely trained slag grade identification model.

[0027] S24. Test the trained model using real muck images; input the real muck images into the model, and the model outputs the predicted muck grades; calculate evaluation metrics such as the accuracy, precision, recall, and F-value of the model by comparing with the real muck grades; further optimize the model according to the evaluation results to improve the performance of the model.

[0028] The beneficial effects of the present invention are as follows:

[0029] 1) The present invention uses the Unity virtual engine to generate a synthetic dataset, effectively solving the problem of insufficient muck image samples, reducing the data annotation cost. At the same time, the dataset is optimized through reinforcement learning, improving the similarity between the synthetic dataset and the real dataset, and providing better data support for the muck grade recognition model.

[0030] 2) The muck grade classification algorithm designed based on deep learning selects a suitable convolutional neural network as the basic network through comprehensive evaluation, and can achieve efficient and accurate muck grade recognition. Compared with traditional methods, the recognition accuracy and efficiency are greatly improved.

[0031] 3) The system and method provide an automated muck grade recognition means for TBM construction, which helps to adjust the tunneling parameters in a timely manner, improve the construction efficiency, reduce the tool wear, and lower the construction risk, having important engineering application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the system framework structure of the present invention;

[0033] Figure 2 It is a schematic diagram of the optimization process of the synthetic dataset based on reinforcement learning of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0035] Embodiment 1, as Figure 1 shown, a TBM muck grade recognition system based on a synthetic dataset and deep learning includes a synthetic dataset and a muck grade recognition system. The synthetic dataset includes:

[0036] Building a virtual scene: Select to use Unity to build a virtual scene; collect 3D models of rock slag; build a conveyor belt model; use C# scripts to implement conveyor belt control; build a virtual camera; set adjustable parameters of the scene; export photos taken by the virtual camera; read the rendering cache of Unity to automatically generate labels for the dataset; export the synthesized dataset;

[0037] Optimizing the virtual scene through reinforcement learning: Obtain a real dataset for rock slag grade recognition; analyze project requirements and select a suitable reinforcement learning algorithm; use image similarity evaluation based on the hash algorithm as the basis for the reward function; formulate the reward function of reinforcement learning to provide timely feedback for the decision-making process; establish a neural network model based on the value function and the reward function; optimize the parameters of the virtual scene through the neural network model; obtain the optimal virtual scene parameters;

[0038] The rock slag grade recognition system includes: dividing the training set and the test set based on the synthesized dataset; comparing the algorithm efficiency and accuracy of different convolutional neural network models and selecting a suitable convolutional neural network model; training the convolutional neural network model of deep learning; optimizing the parameters of the rock slag grade recognition model; solidifying the optimal model parameters; evaluating the grade recognition effect of the model on real rock slag pictures.

[0039] Furthermore, when building the virtual scene, use the python interface to give the initial parameters of the Unity virtual environment. Unity generates the virtual environment and takes pictures through the virtual camera to obtain the synthesized dataset, and compares the differences between the synthesized dataset and the real dataset.

[0040] Furthermore, the optimization of the virtual scene is based on the image similarity calculation program based on the hash algorithm as the basis for the reward function. Give the reward value according to the image similarity difference and update the value function; the agent takes the most valuable behavior according to the value function, updates the parameters of the virtual environment, repeats the training until the set number of times or the image similarity reaches the standard, and solidifies the value function; finally, adjust the parameters of the virtual scene according to the solidified value function to obtain a synthesized dataset with labels similar to the real dataset.

[0041] Furthermore, when the rock slag grade recognition system recognizes real rock slag pictures, the preprocessing of real rock slag pictures includes: cropping the original rock slag image to remove unnecessary redundant backgrounds and reduce interference from irrelevant factors; then performing detail enhancement filtering to make the image details more obvious, while increasing the image contrast and saturation, enhancing the contrast of each part of the picture, and making the rock slag contour clearer;

[0042] The rock slag grade recognition system uses a synthetic dataset to test the accuracy and operation time of each network, comprehensively evaluates the image classification accuracy and algorithm efficiency, and selects a suitable network as the base network for feature extraction. The selected base networks include, but are not limited to: LeNet-5, AlexNet, VGG-16, Inception-v3 as candidate networks;

[0043] The training and optimization of the model by the rock slag grade recognition system specifically include: using the synthetic dataset as the input and the classification grades of large, medium, and small rock slags as the output, constructing a rock slag grade recognition model based on the selected base network, and optimizing the model parameters by repeatedly training the convolutional neural network model, and finally solidifying the optimal model parameters;

[0044] The testing and evaluation of the model by the rock slag grade recognition system specifically include: using real rock slag images to test the trained model, and comprehensively evaluating the model performance using multiple indicators such as accuracy, precision, recall rate, and F-value.

[0045] Example 2, as Figure 2 shown, a TBM rock slag grade recognition method based on a synthetic dataset and deep learning includes the following steps:

[0046] S1. Synthetic dataset generation

[0047] S11. In the Unity3D environment, collect a large number of 3D rock slag models and package them into prefabs, and attach rigidbody components; create a platform and a baffle, attach rigidbody components to them respectively and set them to static, and set up a camera and a light source directly above the platform;

[0048] S12. Set enough generation points on a horizontal plane slightly higher than the platform, and use a C# script to achieve that the rock slag model prefab randomly selects a generation point for instantiation every 0.1s and is destroyed after n*0.1s, where n is the number of rock slag particles on the test bench; perform scaling of 0.8 - 1.2 times on the replicated body during instantiation, name them in sequence and assign rotation characteristics to simulate the actual slag discharge situation of the TBM;

[0049] S13. Use the RenderTexture library to write a script and attach it to the camera. Create a folder and name it in sequence every n*m*0.1s, where m is the number of photos in the folder; render and save the pictures to the corresponding folder every n*0.1s after n*0.1s of running the virtual scene; use the EPPlus plugin to write a script and attach it to the camera, and export the position coordinates, length, width and other related parameters of the rock slag to the excel folder every n*0.1s after n*0.1s of running the virtual scene for convenient subsequent calling;

[0050] In S14, the Python interface uses socket communication to transmit initial parameters to the Unity virtual environment and start the reinforcement learning training. Unity generates a virtual environment and renders images through a virtual camera to obtain a synthetic dataset. The OpenCV library is used to read the images in the synthetic dataset and the real dataset, and the hash algorithm is used to calculate the image hash values. The hash value differences between the synthetic images and the real images are compared, the minimum difference value is selected and averaged to obtain the average difference value that quantifies the differences between the two datasets.

[0051] In S15, the reward value is determined according to the image similarity calculation program based on the hash algorithm to update the value function. The agent selects actions according to the value function to update the virtual environment parameters, and the training process is repeated. When the set number of training times is reached or the image similarity meets the standard, the training is stopped and the value function is solidified.

[0052] Based on the solidified value function, in S16, the Python interface is used again to give the initial parameters of the Unity virtual environment, the action with the largest value is selected from the Q table to adjust the virtual scene parameters, an optimized synthetic dataset is generated, and labels are added to each image to form a labeled dataset.

[0053] S2. Steps for identifying the slag grade

[0054] After obtaining the slag image, in S21, image preprocessing is first performed. The image editing tool or programming library (such as OpenCV) is used to crop the original image to remove the background part irrelevant to the slag in the image. Then, algorithms such as Gaussian filtering and Laplace filtering are applied for detail enhancement filtering to improve the image details. By adjusting the brightness, contrast, and saturation parameters of the image, the contrast of each part of the image is enhanced.

[0055] In S22, the preprocessed image is input into candidate convolutional neural network models such as LeNet-5, AlexNet, VGG-16, and Inception-v3, and the accuracy and operation time of each model are tested using the synthetic dataset. The classification accuracy of each model for the images in the synthetic dataset and the time required for the model to run are recorded, and the model with high classification accuracy and fast operation efficiency is selected as the basic network for slag grade identification after comprehensive evaluation.

[0056] S23. Construct a rock slag grade recognition model with the selected basic network as the framework; use the images in the synthetic dataset as the input, and label them according to the three rock slag grades of large, medium, and small as the output of the model; use a deep learning training framework (such as PyTorch or TensorFlow) to train the model. During the training process, continuously adjust the weights and parameters of the model, and minimize the loss function through the backpropagation algorithm to enable the model to gradually learn the mapping relationship between the rock slag image features and grades; after multiple rounds of training, solidify the optimal model parameters to obtain a well-trained rock slag grade recognition model;

[0057] S24. Use real rock slag images to test the trained model; input the real rock slag images into the model, and the model outputs the predicted rock slag grades; by comparing with the real rock slag grades, calculate evaluation indicators such as the accuracy, precision, recall rate, and F-value of the model; according to the evaluation results, further optimize the model, such as adjusting the model structure, increasing the amount of training data, or adjusting the training parameters, etc., to improve the performance of the model.

[0058] Use the Unity virtual engine to simulate the TBM construction scenario, dynamically generate 3D models of rock slag with different sizes, textures, and lighting conditions through C# scripts, control their physical properties (such as rigid body components, scaling, rotation), and use a virtual camera to capture images to generate diverse synthetic rock slag data; connect Unity and the reinforcement learning framework through a Python interface, input the synthetic images and real images into the similarity calculation module based on the hash algorithm, quantify the differences and generate reward values. The agent iteratively adjusts the virtual scene parameters (such as lighting, model distribution) according to the reward values to optimize the value function until the gap between the synthetic data and the real data domain is minimized. Finally, generate a synthetic dataset with labels, and the labels are automatically annotated by the geometric parameters of the rock slag (length, width, position), significantly reducing the manual annotation cost.

[0059] Perform background cropping, detail enhancement filtering, and contrast adjustment on the input image to highlight the rock slag contour, select the optimal basic network, use the synthetic dataset as the input, and the rock slag grade (large, medium, small) as the label, train the model through the deep learning framework, optimize the parameters using backpropagation, solidify the optimal model, and verify the model performance using real rock slag images, and evaluate through indicators such as accuracy and recall rate. Finally, deploy it to the TBM site to analyze the rock slag images on the conveyor belt in real time, output the grade results, and support the dynamic adjustment of construction parameters.

[0060] It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention. Therefore, in any aspect, the embodiments should be regarded as exemplary and non-restrictive. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

[0061] In addition, it should be understood that although this specification is described in terms of embodiments, not every embodiment only contains an independent technical solution. This narrative manner of the specification is only for clarity. Those skilled in the art should regard the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A TBM slag grade recognition system based on synthetic datasets and deep learning, comprising a synthetic dataset and a slag grade recognition system, characterized in that the synthetic dataset includes: Building a virtual scenario: Select to use unity to build a virtual scenario; collect 3D models of slag; build a conveyor belt model; use C# scripts to implement conveyor belt control; build a virtual camera; set adjustable parameters of the scenario; export the photos taken by the virtual camera; read the unity rendering cache to automatically generate labels for the dataset; export the synthetic dataset; Optimizing the virtual scenario through the method of reinforcement learning: Obtain the real dataset for slag grade recognition; analyze project requirements and select a suitable reinforcement learning algorithm; use the image similarity evaluation based on the hash algorithm as the basis for the reward function; formulate the reward function of reinforcement learning to provide timely feedback for the decision-making process; establish a neural network model according to the value function and the reward function; optimize the parameters of the virtual scenario through the neural network model; obtain the optimal virtual scenario parameters; The slag grade recognition system includes: dividing the training set and the test set based on the synthetic dataset; comparing the algorithm efficiency and accuracy of different convolutional neural network models, and selecting a suitable convolutional neural network model; training the convolutional neural network model of deep learning; optimizing the parameters of the slag grade recognition model; solidifying the optimal model parameters; evaluating the grade recognition effect of the model on real slag pictures.

2. The TBM rock slag grade recognition system according to claim 1, characterized in that: The building of the virtual scenario uses the python interface to give the initial parameters of the Unity virtual environment. Unity generates the virtual environment and takes pictures through the virtual camera to obtain the synthetic dataset, and compares the synthetic dataset with the real dataset for differences.

3. The TBM rock slag grade identification system according to claim 1, wherein: The optimization of the virtual scenario is based on the image similarity calculation program based on the hash algorithm as the basis for the reward function. The reward value is given according to the image similarity difference, and the value function is updated; the agent takes the most valuable behavior according to the value function, updates the virtual environment parameters, repeats the training until the set number of times or the image similarity reaches the standard, and solidifies the value function; finally, adjusts the virtual scenario parameters according to the solidified value function to obtain a synthetic dataset with labels similar to the real dataset.

4. The TBM muck grade identification system according to claim 1, characterized in that: When the slag grade recognition system recognizes real slag pictures, the preprocessing of the real slag pictures includes: cropping the original slag image to remove unnecessary redundant backgrounds and reduce interference from irrelevant factors; then performing detail enhancement filtering to make the image details more obvious, while increasing the image contrast and saturation, enhancing the contrast of each part of the picture, and making the slag contour clearer; The slag grade recognition system uses the synthetic dataset to test the accuracy and operation time of each network, comprehensively evaluates the image classification accuracy and algorithm efficiency, and selects a suitable network as the basic network for feature extraction. The selected basic networks include but are not limited to: LeNet-5, AlexNet, VGG-16, Inception-v3 as candidate networks; The training and optimization of the rock slag grade recognition system for the model specifically include: using the synthetic dataset as the input and the classification grades of large, medium, and small rock slag as the output, constructing a rock slag grade recognition model based on the selected basic network, optimizing the model parameters by repeatedly training the convolutional neural network model, and finally solidifying the optimal model parameters; The testing and evaluation of the rock slag grade recognition system for the model specifically include: using real rock slag images to test the trained model, and comprehensively evaluating the model performance using multiple indicators such as accuracy, precision, recall rate, and F value.

5. A method for identifying TBM slag grades based on a synthetic dataset and deep learning, characterized in that: It includes the following steps: S1. Generation of synthetic dataset S11. In the Unity3D environment, collect a large number of 3D rock slag models and package them into prefabs, and attach rigidbody components; create a platform and a baffle, attach rigidbody components to them respectively and set them as static, and set up a camera and a light source directly above the platform; S12. Set enough generation points on a horizontal plane slightly higher than the platform, and use a C# script to achieve that the rock slag model prefab randomly selects a generation point for instantiation every 0.1s and is destroyed after n * 0.1s, where n is the number of rock slag particles on the test bench; perform a 0.8 - 1.2 times scaling on the replicated object during instantiation, name it in sequence and endow it with rotational characteristics to simulate the actual slag discharge situation of the TBM; S13. Use the RenderTexture library to write a script and attach it to the camera. Create a folder and name it in sequence every n * m * 0.1s, where m is the number of photos in the folder; render and save the images to the corresponding folder every n * 0.1s after n * 0.1s of running the virtual scene; use the EPPlus plugin to write a script and attach it to the camera, and export the position coordinates, length, width and other related parameters of the rock slag to the excel folder every n * 0.1s after n * 0.1s of running the virtual scene for convenient subsequent call; S14. The python interface uses socket communication to transmit initial parameters to the Unity virtual environment and start the reinforcement learning training; Unity generates a virtual environment and obtains a synthetic dataset by rendering images through a virtual camera. Use the OpenCV library to read the images in the synthetic dataset and the real dataset, calculate the image hash value using the hash algorithm, compare the hash value differences between the synthetic image and the real image, select the minimum difference value and calculate the average to obtain the average difference value quantifying the differences between the two datasets; S15. Determine the reward value according to the image similarity calculation program based on the hash algorithm and update the value function; the agent selects actions according to the value function to update the virtual environment parameters and repeats the training process; when the set number of training times is reached or the image similarity meets the standard, stop the training and solidify the value function; S16. Based on the solidified value function, use the python interface to give the initial parameters of the Unity virtual environment again, select the action with the largest value from the Q table to adjust the virtual scene parameters, generate an optimized synthetic dataset, and add labels to each image to form a labeled dataset; S2. Rock slag grade recognition steps After obtaining the rock slag image, first perform image preprocessing; use an image editing tool or programming library to crop the original image and remove the background part irrelevant to the rock slag in the image; then apply algorithms such as Gaussian filtering and Laplacian filtering for detail enhancement filtering to improve the image details; enhance the contrast of each part of the image by adjusting the brightness, contrast, and saturation parameters of the image. S22. Input the preprocessed image into the candidate convolutional neural network models of LeNet-5, AlexNet, VGG-16, and Inception-v3, and use the synthetic dataset to test the accuracy and operation time of each model; record the classification accuracy of each model for the images in the synthetic dataset and the time required for the model to run, and select the model with high classification accuracy and fast operation efficiency as the basic network for rock slag grade recognition after comprehensive evaluation. S23. Construct a rock slag grade recognition model with the selected basic network as the framework; use the images in the synthetic dataset as the input and label them according to the three rock slag grades of large, medium, and small as the output of the model; use a deep learning training framework to train the model. During the training process, continuously adjust the weights and parameters of the model, and minimize the loss function through the backpropagation algorithm to enable the model to gradually learn the mapping relationship between the rock slag image features and grades; after multiple rounds of training, solidify the optimal model parameters to obtain a well-trained rock slag grade recognition model. S24. Use real rock slag images to test the trained model; input the real rock slag images into the model, and the model outputs the predicted rock slag grades; calculate the evaluation indicators of the model's accuracy, precision, recall rate, and F value by comparing with the real rock slag grades; further optimize the model according to the evaluation results to improve the performance of the model.

Images