Tunnel hole slag image intelligent identification method and system based on multi-task deep learning

Abstract

The invention discloses a tunnel hole slag image intelligent identification method and system based on multi-task deep learning, and relates to the technical field of crossing of hydraulic engineering and artificial intelligence, and the method comprises the steps: obtaining a hole slag image and shared data, constructing an original data set, preprocessing the original data set by using a dynamic adaptive data enhancement strategy and a focus loss function to obtain an enhanced data set; constructing a multi-level classification deep learning model of a multi-branch output structure, respectively outputting probability distribution results of different classifications, and setting a loss function and a loss weight of the multi-level classification deep learning model; and performing optimization training on the multi-stage deep learning model, inputting an enhanced data set generated in real time into the multi-stage classification deep learning model after optimization training, and outputting drilling and blasting parameter recommendation based on a lithology identification result. According to the method, the hierarchical classification model with multi-branch output is constructed, and construction method identification and specific lithology classification can be realized at the same time.

Description

Technical Field

[0001] This invention relates to the field of interdisciplinary technology of water conservancy engineering and artificial intelligence, and in particular to a method and system for intelligent recognition of tunnel muck images based on multi-task deep learning. Background Technology

[0002] The construction of hydraulic tunnels generates a large amount of excavated soil. Traditional methods of dumping excavated soil not only occupy a significant amount of land resources but also pose substantial ecological and environmental risks. Statistics show that the total amount of excavated soil from water conservancy projects nationwide exceeds 600 million cubic meters, occupying an average of over 28,000 hectares of arable land annually—equivalent to the ecological loss of approximately 40 West Lakes' water area each year. Furthermore, it easily triggers geological disasters such as mudslides and landslides, posing a serious threat to regional sustainable development. On the other hand, the demand for aggregates in hydraulic engineering construction continues to grow, exceeding 6 billion tons. Natural sand and gravel resources are increasingly depleted due to over-exploitation, exacerbating the contradiction of non-renewable resources. Against this backdrop, a prominent contradiction has emerged: "large-scale generation of excavated soil—strong demand for construction materials."

[0003] Transforming cave debris into building materials offers significant economic and environmental benefits, but its resource utilization faces severe technical challenges. Cave debris exhibits complex lithology, uneven particle size distribution, large fluctuations in mechanical properties, and strong gradation dispersion. Without scientific sorting and processing, it not only fails to meet the gradation requirements of hydraulic engineering building materials but also directly affects the strength, durability, and stability of concrete, potentially introducing hidden engineering quality problems. Therefore, accurate and rapid identification and classification of cave debris is a prerequisite for achieving its large-scale, high-value resource utilization and a key link in promoting green construction and intelligent engineering development.

[0004] However, traditional methods for identifying tunnel debris mainly rely on manual judgment, which is inefficient and highly subjective, making it difficult to meet the current engineering requirements for high real-time performance and accuracy. In recent years, with the rapid development of artificial intelligence technology, especially the successful application of computer vision and intelligent sensing technology in rock mass identification, particulate material analysis, and blasting optimization, a brand-new solution has been provided for the intelligent identification of tunnel debris. For example, patent CN116912300A discloses a "Tunnel Blasting Method Based on Intelligent Identification of Surface Cracks and Internal Rock Properties," which uses three-dimensional laser scanning to obtain point cloud data of the tunnel face, combines deep learning for joint identification and blasting numerical simulation, and then optimizes the layout of blast holes and the design of charge quantity, achieving refined control of the blasting process; patent CN112801035B provides a "Knowledge and Data Dual-Driven Onboard Intelligent Rock Properties Identification Method and System," which integrates multi-angle images and element content information, combines convolutional neural networks and transfer learning to achieve rock properties identification, and can be mounted on tunneling equipment to assist in real-time decision-making.

[0005] Although the above methods have achieved good results in specific applications, they still have the following limitations: First, they rely heavily on high-cost sensing equipment (such as 3D laser scanners and X-ray fluorescence spectrometers), which are complex to deploy and difficult to promote on a large scale in resource-constrained construction environments. Second, the identification function is relatively simple, failing to integrate construction method judgment and lithology classification into a unified hierarchical structure, and unable to output an overall identification result of "construction method → ​​mechanical subclass / lithology → drilling and blasting parameter recommendation", thus limiting its direct application value in the resource utilization of tunnel cuttings. Third, there is insufficient attention to the problem of sample class imbalance, especially the poor identification performance of minority class samples generated by mechanical construction methods, which affects the model's generalization ability in real-world scenarios.

[0006] Therefore, there is an urgent need for a low-cost, high-efficiency, multi-layer classification method that can simultaneously achieve intelligent identification of construction methods and lithology, in order to improve the accuracy and engineering applicability of tunnel muck identification and support real-time decision-making and resource utilization at the construction site. Summary of the Invention

[0007] Therefore, it is necessary to provide a method and system for intelligent recognition of tunnel muck images based on multi-task deep learning to address the aforementioned technical problems.

[0008] In a first aspect, the present invention provides an intelligent recognition method for tunnel muck images based on multi-task deep learning, comprising: S1. Obtain images of the hole debris and shared data, construct the original dataset, and preprocess the original dataset using a dynamic adaptive data augmentation strategy and a focus loss function to obtain the augmented dataset; S2. Based on the preset backbone feature extraction network and pre-trained weights, construct a multi-level classification deep learning model with a multi-branch output structure, output the probability distribution results of different classifications respectively, and set the loss function and loss weight of the multi-level classification deep learning model. S3. Based on preset multi-type callback functions, optimize and train the multi-level deep learning model, and input the real-time generated augmented dataset into the optimized multi-level classification deep learning model to output drilling and blasting parameter recommendations based on lithology identification results.

[0009] Furthermore, by acquiring images of the hole debris and sharing data, a raw dataset is constructed. The raw dataset is then preprocessed using a dynamic adaptive data augmentation strategy and a focus loss function to obtain an augmented dataset, including: S11. Obtain images of the slag generated during tunnel construction, and combine them with the shared data provided by the rock, mineral and fossil resource sharing library to construct a three-level layered original dataset. The three-level layered structure includes construction methods, mechanical subclasses or lithology, and recommended drilling and blasting parameters. S12. Traverse the directory to obtain all image file paths and generate a three-level one-hot encoded label vector; S13. A dynamic adaptive data augmentation strategy based on the target sample size is used to enhance the images of slag from different types of construction methods, generating diverse sample images. S14. Based on a preset focus loss function, train the sample images corresponding to construction methods with an absolute number greater than a preset threshold to generate trained and adjusted sample images; then merge the enhanced sample images with the original cavity images to form the final augmented dataset. S15. Divide the augmented dataset into training and validation sets according to a preset ratio, and build a high-efficiency data pipeline based on an efficient data input pipeline building tool to support parallel decoding, caching, and prefetch optimization.

[0010] Furthermore, based on a dynamic adaptive data augmentation strategy using the target sample size, images of caving debris from different construction methods are enhanced to generate diverse sample images, including: S131. Calculate the feature map of each cavity muck image by perceiving the visual characteristics of the image, extract the texture complexity features and key area saliency map of the cavity muck image, and combine them with the preset difficulty coefficient corresponding to the construction method of the cavity muck image to construct a comprehensive dynamic enhanced difficulty score. S132. Construct a lightweight policy network, using the losses of the original samples, the most augmented samples, and the non-augmented samples in the current training cycle as state inputs, and combining the augmentation difficulty score, dynamically output the augmentation operation magnitude customized for each training sample through the superior actor-critic algorithm. S133. Based on the enhancement operation amplitude, the enhancement operation is performed cyclically on each cavity debris image, and the cavity debris images at the end of the cycle are integrated as diverse sample images. The random variation operation includes random horizontal flipping, random brightness adjustment, random contrast adjustment, random saturation adjustment, random 90° rotation, image scaling to a uniform size, and pixel value normalization.

[0011] Furthermore, the formula for calculating the increased difficulty score is as follows: ; ; In the formula, Indicates the difficulty score; indicates the weighting coefficients of different factors; represents the significance plot. S The global average; The preset difficulty coefficient indicates the construction method type to which the caving debris image belongs; The texture complexity score represents the image of the hole debris. N This represents the total number of local blocks in the cavity image; The first one in the image of the hole debris i A local area; This represents the function for calculating the standard deviation of a given image patch.

[0012] Furthermore, a lightweight policy network is constructed, using the losses of the original samples, the most augmented samples, and the non-augmented samples in the current training epoch as state inputs, and combining them with the augmentation difficulty score. The superior actor-critic algorithm dynamically outputs the augmentation operation magnitude customized for each training sample, including: S1321. Set the training period, collect the historical loss values ​​fed back by the multi-level classification deep learning model to construct the state vector, and concatenate the state vector with the enhancement difficulty score of the hole debris image to merge it into the input state of the policy network. S1322. Construct a lightweight neural network as a policy network, and introduce an activation function in the output layer to generate continuous enhancement magnitude values ​​between 0 and 1 for each available image enhancement operation. S1323. Construct a value network to evaluate the current state, use a policy network to output the augmentation magnitude according to the current state, and calculate the reward signal based on the performance feedback of the target model after training on the augmented sample images. S1324. Use the dominance function to guide the parameter update direction of the policy network. Calculate the difference between the reward obtained by taking an actual action in a specific state and the baseline value of that state predicted by the value network through the dominance function, and dynamically output the magnitude of each enhancement operation.

[0013] Furthermore, based on the pre-defined backbone feature extraction network and pre-trained weights, a multi-level classification deep learning model with a multi-branch output structure is constructed to output the probability distribution results for different classifications. The loss function and loss weights of the multi-level classification deep learning model are defined as follows: S21. A 128-dimensional fully connected layer is connected, and a non-linear feature mapping is introduced using the ReLU activation function. A random deactivation layer and an output layer with three neurons are also introduced to output the probability distribution of different types of construction methods, forming the first branch output structure of the multi-level classification deep learning model. S22. A 128-dimensional fully connected layer and a random deactivation layer are connected, and an output layer with two neurons is introduced to output the probability distribution of different types of mechanical subclasses, forming the second branch output structure of the multi-level classification deep learning model. S23. Connect a 128-dimensional fully connected layer and a random deactivation layer, and introduce an output layer with three neurons to output the probability distribution representing different types of lithology, forming the second branch output structure of the multi-level classification deep learning model. S24. A dynamic loss weight balancing strategy is adopted to construct a multi-task loss function, and the weights are automatically adjusted according to the uncertainty of multi-tasks to achieve adaptive optimization of the loss function.

[0014] Furthermore, a dynamic loss weight balancing strategy is adopted to construct a multi-task loss function, and the weights are automatically adjusted according to the uncertainty of multi-tasks, including: S241. Set task-specific loss functions for the output structures of different branches of the multi-level classification deep learning model, and initialize the associated learnable noise parameters. The first branch output structure adopts the focus loss function, and the second and third branch output structures adopt the cross-entropy loss function. S242. During the forward propagation process of each training batch, based on the current noise parameter value, the adaptive weight of the loss of each task is dynamically calculated according to the uncertainty weighting principle. S243. The loss values ​​of each task are weighted and summed with the adaptive weights, and a regularization term is added to prevent the noise parameters from increasing indefinitely during training, so as to obtain the total loss value after fusion. S244. During backpropagation, the gradient of the total loss value with respect to the model's master parameters and the noise parameters of each task is calculated. The optimizer is used to update the model's master parameters and the noise parameters simultaneously, so as to realize the training iteration of the multi-level classification deep learning model that automatically adjusts the weights according to the current learning status of each task.

[0015] Furthermore, based on preset multi-type callback functions, the optimization training of multi-level deep learning models includes: S31. Divide the training process of the multi-level classification deep learning model into several stages. Use early stopping to monitor the change of validation loss on the validation set at the end of each stage. If the validation loss does not improve within a continuous patience value, terminate the training and restore the best weight in the monitoring index. S32. Use model checkpoints to monitor the validation loss of each stage. When each stage ends, if the validation loss of the current stage is lower than the validation loss of all previous stages, the optimal multi-level classification deep learning model is saved to the specified file. S33. Use a logger to record the training loss, validation loss, training accuracy, and validation accuracy at each stage, automatically append them to the same CSV file, and plot the loss vs. accuracy curve. S34. Establish multi-task performance and engineering physical constraints, perceive the tasks and loss values ​​of multi-level deep learning models in real time for verification, and trigger loss correction based on the verification results.

[0016] Furthermore, multi-task performance and engineering physical constraints are established, and the tasks and loss values ​​of the multi-level deep learning model are evaluated in real time. Based on the validation results, loss correction is triggered, including: S341. Real-time calculation of the F1 score of the output structure of the three branches on the validation set of the multi-level task deep learning model. Calculate the absolute value of the difference between the F1 scores of any two tasks as a measure of the performance balance between tasks. Based on the preset performance difference threshold, if the absolute value of the difference between the F1 scores of any pair of tasks exceeds the performance difference threshold, a task imbalance warning is triggered. S342. Upon receiving a task imbalance warning signal, automatically activate a targeted learning rate adjustment strategy to reduce the learning rate of the backbone feature extraction network shared by all tasks and proportionally increase the learning rate of the branch network to which the imbalanced task belongs. S343. Establish an engineering physics rule base, set the logical relationships between different construction methods, machinery subcategories, or rock types and drilling and blasting parameter recommendations, and input the model's prediction results into the engineering physics rule base for logical compliance verification during the training process.

[0017] Secondly, the present invention also provides an intelligent recognition system for tunnel muck images based on multi-task deep learning, the system comprising: The image acquisition and processing module is used to acquire images of the cavitation debris and shared data, construct the original dataset, and preprocess the original dataset using a dynamic adaptive data augmentation strategy and a focus loss function to obtain the augmented dataset. The model building and optimization module is used to construct a multi-level classification deep learning model with a multi-branch output structure based on a preset backbone feature extraction network and pre-trained weights, output the probability distribution results of different categories respectively, and set the loss function and loss weight of the multi-level classification deep learning model. The model training output module is used to optimize and train a multi-level deep learning model based on preset multi-type callback functions, and input the real-time generated augmented dataset into the optimized multi-level classification deep learning model to output drilling and blasting parameter recommendations based on lithology identification results.

[0018] The beneficial effects of this invention are as follows: 1. This invention possesses hierarchical recognition and decision-making capabilities. Using EfficientNet-B4 as the backbone feature extraction network, it constructs a hierarchical classification model with multi-branch outputs, which can simultaneously identify construction methods (mechanical methods or drill-and-blast methods) and classify specific lithologies (granite, limestone, or sandstone). For the first time, it achieves conditional multi-level recognition of "construction method - mechanical subclass / lithology - drill-and-blast parameter recommendation". The recognition results can directly guide the adjustment of drill-and-blast construction parameters and improve the utilization rate of tunnel cuttings resources.

[0019] 2. This invention achieves data imbalance optimization. By introducing a dynamic adaptive data augmentation strategy and a focus loss function, it effectively alleviates the problem of sample class imbalance and significantly improves the recognition performance of minority class samples. With the help of the "dynamic adaptive data augmentation + focus loss" strategy, the accuracy of the optimal model construction method is 99.0%, and the lithology classification accuracy is 99.0%, solving the problem of small sample class recognition.

[0020] 3. This invention achieves high precision and real-time performance, namely, based on shared feature extraction and multi-branch design of EfficientNet-B4, the best model achieves a construction method accuracy of 99.0% and a lithology classification accuracy of 99.0% on the validation set, and the single image inference time is fast, supporting real-time decision-making at the construction site.

[0021] 4. This invention enables low-cost and easy deployment. It only requires ordinary cameras to collect images, and the model can be exported as a tunnel_muck_classifier.keras file, which can be deployed on edge computing devices (such as industrial tablets), greatly reducing the cost and threshold of engineering applications. Attached Figure Description

[0022] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings: Figure 1 This is a flowchart of a method for intelligent recognition of tunnel muck images based on multi-task deep learning according to an embodiment of the present invention; Figure 2 This is a hierarchical directory structure diagram of the original dataset in a method for intelligent recognition of tunnel muck images based on multi-task deep learning according to an embodiment of the present invention. Figure 3 This is a flowchart of the model recognition process in a method for intelligent recognition of tunnel muck images based on multi-task deep learning according to an embodiment of the present invention. Figure 4 This is a block diagram illustrating the principle of a tunnel muck image intelligent recognition system based on multi-task deep learning according to an embodiment of the present invention. Figure 5 This is a training and verification accuracy and loss rate curve of the construction method according to an embodiment of the present invention; Figure 6 This is a graph showing the accuracy and loss rate of training and validation for the mechanical subclass according to an embodiment of the present invention; Figure 7 This is a graph showing the accuracy and loss rate of drill-and-blast method lithology classification training and verification according to an embodiment of the present invention. Figure 8 These are test images of the shield tunneling method according to an embodiment of the present invention.

[0023] The reference numerals are: 1. Image acquisition and processing module; 2. Model building and optimization module; 3. Model training output module. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0025] Please see Figure 1 This paper presents an intelligent recognition method for tunnel muck images based on multi-task deep learning. The method includes: S1. Obtain images of the hole debris and shared data to construct the original dataset. Then, use a dynamic adaptive data augmentation strategy and a focus loss function to preprocess the original dataset to obtain the augmented dataset.

[0026] In the description of this invention, hole debris images and shared data are acquired to construct an original dataset. The original dataset is then preprocessed using a dynamic adaptive data augmentation strategy and a focus loss function to obtain an augmented dataset, including: S11. Obtain images of tunnel debris generated during tunnel construction, and combine them with shared data provided by the rock, mineral and fossil resource sharing library to construct a three-level layered original dataset. The three-level layered structure includes construction methods, mechanical subclasses or lithology, and recommended drilling and blasting parameters.

[0027] Specifically, the original dataset is mainly composed of images of tunnel debris generated during tunnel construction, supplemented by the National Rock, Mineral and Fossil Resource Sharing Database. It is constructed into a three-level layered structure according to "construction method → ​​mechanical subclass / lithology → recommended drilling and blasting parameters". The construction methods are divided into TBM, shield tunneling, and drilling and blasting. The drilling and blasting method is further divided into three lithologies: granite, limestone, and sandstone.

[0028] S12. Traverse the directory to obtain all image file paths and generate a three-level one-hot encoded label vector.

[0029] Specifically, the construction method tags are: mechanical method 0, drill and blast method 1; the mechanical method sub-category tags are: TBM method 0, shield tunneling method 1; and the lithology tags are: granite 0, limestone 1, sandstone 2.

[0030] S13. A dynamic adaptive data augmentation strategy based on the target sample size is used to enhance the images of slag from different types of construction methods, generating diverse sample images.

[0031] Specifically, to address the sample imbalance problem of TBM, shield tunneling, and drill-and-blast methods, a dynamic adaptive data augmentation strategy based on the target sample size is adopted: by setting an augmentation target, the required number of augmentations is dynamically calculated according to the category of each original image and its preset difficulty coefficient (initially 1.0). For the calculated augmentation operation magnitude for each image, seven random transformation operations are cyclically executed: random horizontal flipping, random brightness adjustment, random contrast adjustment, random saturation adjustment, random 90° rotation, image scaling to a uniform size of 224×224, and pixel value normalization to the [0, 1] interval, generating diverse new samples.

[0032] In the description of this invention, a dynamic adaptive data augmentation strategy based on a target sample size is used to enhance images of caving debris from different types of construction methods, generating diverse sample images including: S131. Calculate the feature map of each cavity muck image by perceiving the visual characteristics of the image, extract the texture complexity features and key area saliency map of the cavity muck image, and combine them with the preset difficulty coefficient corresponding to the construction method of the cavity muck image to construct a comprehensive dynamic enhanced difficulty score.

[0033] In the description of this invention, the formula for calculating the enhanced difficulty score is as follows: ; ; In the formula, Indicates the difficulty score; indicates the weighting coefficients of different factors; represents the significance plot. S The global average; The preset difficulty coefficient indicates the construction method type to which the caving debris image belongs; The texture complexity score represents the image of the hole debris. N This represents the total number of local blocks in the cavity image; The first one in the image of the hole debris i A local area; This represents the function for calculating the standard deviation of a given image patch.

[0034] S132. Construct a lightweight policy network, using the losses of the original samples, the most augmented samples, and the non-augmented samples in the current training cycle as state inputs, and combining them with the augmentation difficulty score, dynamically outputting the augmentation operation magnitude customized for each training sample through the superior actor-critic algorithm.

[0035] In the description of this invention, a lightweight policy network is constructed, using the losses of the original samples, the most boosted samples, and the non-boosted samples in the current training epoch as state inputs, and combining them with boosting difficulty scores. The network dynamically outputs a boosting operation magnitude customized for each training sample through the dominant actor-critic algorithm, including: S1321. Set the training period, collect the historical loss values ​​fed back by the multi-level classification deep learning model to construct the state vector, and concatenate the state vector with the enhancement difficulty score of the hole debris image to merge it into the input state of the policy network.

[0036] Specifically, by setting a fixed training period, such as each epoch, the historical loss values ​​fed back by the multi-level classification deep learning model during training are systematically collected, including the original sample loss, the maximum augmented sample loss, and the non-augmented sample loss. These scalar data are then used to construct a multi-dimensional state vector. This state vector reflects the overall performance and stability of the model in the current training phase. Subsequently, it is concatenated with a pre-calculated hole fragment image augmentation difficulty score and merged into a unified input state for the policy network.

[0037] The principle lies in reflecting the model's intrinsic training dynamics through loss changes and combining them with the external characteristics of the data to form an information-rich state representation. This enables the policy network to fully perceive the current training progress and challenges, providing a comprehensive and structured basis for subsequent decisions. It avoids the bias that may be caused by relying on a single signal and enhances the robustness and representativeness of the state input.

[0038] S1322. Construct a lightweight neural network as a policy network, and introduce an activation function in the output layer to generate a continuous enhancement magnitude value between 0 and 1 for each available image enhancement operation.

[0039] Specifically, a lightweight neural network is constructed as the policy network (Actor), whose structure employs a multilayer perceptron or a simplified convolutional module, with the core objective of maintaining a low number of parameters and computational complexity. The network receives the concatenated state vector generated in step S1321 as input, and performs feature transformation and abstraction through several hidden layers (such as fully connected layers and ReLU activation functions). Finally, a neuron is set up in the output layer for each available image enhancement operation, and a sigmoid is used as the activation function to ensure that each neuron outputs a continuous value between 0 and 1, which represents the proposal magnitude of the corresponding enhancement operation.

[0040] The principle is to leverage the powerful nonlinear fitting ability of neural networks to learn the mapping relationship from a complex state space to an action space (amplification magnitude). The sigmoid function naturally limits the output within a reasonable range, facilitating control of the augmentation intensity. The technical effect of this step is to generate fine-grained, fine-tunable augmentation strategies, enabling each training sample to receive a customized augmentation scheme, rather than a one-size-fits-all approach, thus more accurately balancing the needs of data augmentation and model training.

[0041] S1323. Construct a value network to evaluate the current state, use a policy network to output the augmentation magnitude based on the current state, and calculate the reward signal based on the performance feedback of the target model after training on the augmented sample images.

[0042] Specifically, a value network (Critic) is constructed to evaluate the value of a state. This network typically shares some of the underlying feature extraction layers with the policy network or is constructed independently in parallel.

[0043] The value network receives the same state input and aims to output a scalar value representing the long-term expected reward of the current state. Simultaneously, the policy network is applied based on the magnitude of the augmentation of the current state output to generate augmented sample images, which are then fed into the target model for one round of training or inference. Subsequently, an immediate reward signal, such as the magnitude of loss reduction or accuracy improvement, is calculated based on the performance change of the target model on the validation set or the augmented sample.

[0044] The principle behind this approach borrows from the division of labor and collaboration mechanism in the actor-critic framework. The critic (value network) is responsible for evaluating the quality of the state, providing a benchmark for the actor (policy network)'s decisions. The reward signal, as environmental feedback, directly measures the effectiveness of the augmentation actions taken. This step establishes a closed-loop feedback system, allowing the quality of the augmentation strategy to be quantified by the actual performance improvement of the target model. This tightly links the data augmentation process with the final model's optimization objective, guiding the policy network to learn in a direction that improves model performance.

[0045] S1324. Use the dominance function to guide the parameter update direction of the policy network. Calculate the difference between the reward obtained by taking an actual action in a specific state and the baseline value of that state predicted by the value network through the dominance function, and dynamically output the magnitude of each enhancement operation.

[0046] Specifically, the advantage function is used to guide the parameter update direction of the policy network. The advantage function is usually calculated as the difference between the actual reward (or estimated reward) obtained by taking an actual action in a specific state and the baseline value of that state predicted by the value network.

[0047] When updating parameters, the gradient update magnitude of the policy network will be determined by the magnitude and direction of the advantage value: when the advantage value is positive, it indicates that the current action is better than the average level, and the policy should strengthen in the direction of that action; otherwise, it will weaken.

[0048] The principle is to introduce an advantage function, replacing the benchmark in the policy gradient from a simple cumulative reward to an "advantage" relative to the average level, which significantly reduces the variance of the policy gradient estimation, making the learning process more stable and efficient. This achieves targeted optimization of the policy, ensuring that the adjustment of the augmentation magnitude always aims to improve model performance. Furthermore, due to the reduction in variance, the convergence process of the entire policy network is accelerated, and the adaptability to complex training dynamics is improved.

[0049] S133. Based on the enhancement operation amplitude, the enhancement operation is performed cyclically on each cavity debris image, and the cavity debris images at the end of the cycle are integrated as diverse sample images. The random variation operation includes random horizontal flipping, random brightness adjustment, random contrast adjustment, random saturation adjustment, random 90° rotation, image scaling to a uniform size, and pixel value normalization.

[0050] S14. Based on a preset focus loss function, train the sample images corresponding to construction methods with an absolute number greater than a preset threshold to generate trained and adjusted sample images. Then, merge the enhanced sample images with the original cavity images to form the final augmented dataset. Specifically, during the training process, the absolute number of samples from the drill-and-blast method is still relatively large.

[0051] The standard loss function still dominates the direction of gradient descent. Therefore, this invention introduces a focal loss function, whose mathematical expression is: In the formula, Represents the focus loss function; This represents the probability that the model predicts belongs to the true category; This represents the balancing factor, used to adjust the weights in positive and negative samples; This represents the modulation factor, which is used to reduce the loss contribution of easily separable samples.

[0052] S15. Divide the augmented dataset into training and validation sets according to a preset ratio, and build a high-efficiency data pipeline based on an efficient data input pipeline building tool to support parallel decoding, caching, and prefetch optimization.

[0053] Therefore, this invention addresses the class imbalance problem through a combination of dynamic adaptive data augmentation and a focus loss function. The augmented images are merged with the original images to form the final training dataset. The system automatically divides the training and validation sets in an 8:2 ratio, constructing a high-efficiency data pipeline based on the TensorFlow tf.data API, supporting parallel decoding, caching, and prefetch optimization. This significantly improves data loading efficiency and avoids I / O (input and output) bottlenecks during training.

[0054] S2. Based on the preset backbone feature extraction network and pre-trained weights, construct a multi-level classification deep learning model with a multi-branch output structure, output the probability distribution results of different classifications respectively, and set the loss function and loss weight of the multi-level classification deep learning model.

[0055] In the description of this invention, a multi-level classification deep learning model with a multi-branch output structure is constructed based on a preset backbone feature extraction network and pre-trained weights. This model outputs probability distribution results for different classifications. The loss function and loss weights of the multi-level classification deep learning model are defined as follows: S21. A 128-dimensional fully connected layer is connected, and a nonlinear feature map is introduced using the ReLU activation function. A random deactivation layer and an output layer with three neurons are also introduced to output the probability distribution of different types of construction methods, forming the first branch output structure of the multi-level classification deep learning model.

[0056] S22. A 128-dimensional fully connected layer and a random deactivation layer are connected, and an output layer with two neurons is introduced to output the probability distribution of different types of mechanical subclasses, forming the second branch output structure of the multi-level classification deep learning model.

[0057] S23. A 128-dimensional fully connected layer and a random deactivation layer are connected, and an output layer with three neurons is introduced to output the probability distribution of different types of lithology, forming the second branch output structure of the multi-level classification deep learning model.

[0058] S24. A dynamic loss weight balancing strategy is adopted to construct a multi-task loss function, and the weights are automatically adjusted according to the uncertainty of multi-tasks to achieve adaptive optimization of the loss function.

[0059] Specifically, this invention uses EfficientNet-B4 as the backbone feature extraction network, loads ImageNet pre-trained weights, unfreezes the last 20 layers for fine-tuning, and freezes the statistics of the batch normalization layers. Its constructed multi-branch output structure is as follows: The first branch (construction method classification) uses a 128-dimensional fully connected layer with the ReLU activation function: f(x) = max(0, x) to introduce a non-linear feature map; then a random deactivation layer with a dropout rate of 0.5 is applied to prevent overfitting; the final output layer has 3 neurons and uses the Softmax activation function to output the probability distribution representing the three construction methods.

[0060] The second branch (machine subclass classification): also incorporates a 128-dimensional fully connected layer (ReLU activation) and a random deactivation layer with a dropout rate of 0.5; the final output layer consists of 2 neurons, using the Softmax activation function, and outputs the probability distribution representing the two machine subclasses. To ensure the stability of numerical computation, the output layer of this branch explicitly specifies the data type as float (floating-point number).

[0061] The third branch (lithological classification): also incorporates a 128-dimensional fully connected layer (ReLU activation) and a random deactivation layer with a dropout rate of 0.5; the final output layer consists of 3 neurons, employing the Softmax activation function, and outputs a probability distribution representing the three lithological types. To ensure the stability of numerical computation, the output layer of this branch explicitly specifies the data type as float (floating-point number).

[0062] The output structures of the three branches mentioned above correspond to construction methods, machinery subclasses, and lithological classification probability distributions, respectively, and do not directly represent classification accuracy. Classification accuracy is a performance metric calculated by comparing the predicted results with the true labels during the model evaluation phase.

[0063] In the description of this invention, the multi-task loss function is constructed using a dynamic loss weight balancing strategy, and the weights are automatically adjusted according to the uncertainty of the multi-task process, including: S241. Set task-specific loss functions for the output structures of different branches of the multi-level classification deep learning model, and initialize the associated learnable noise parameters. The first branch output structure adopts the focus loss function, and the second and third branch output structures adopt the cross-entropy loss function.

[0064] Specifically, task-specific loss functions are set for the output structures of different branches of the multi-level classification deep learning model, and the associated learnable noise parameters are initialized.

[0065] Specifically, this includes configuring a focal loss function for the first branch output structure (construction method classification) to address the class imbalance problem, and configuring a cross-entropy loss function for the second branch output structure (mechanical subclass classification) and the third branch output structure (lithology classification) to meet the needs of multi-classification tasks.

[0066] Task-specific loss functions capture the unique optimization objectives of different tasks. For example, focus loss reduces the weight of easily classified samples by using a modulation factor, allowing the model to focus on difficult-to-classify samples, while the noise parameter σ can be learned. t This is used to quantify the random uncertainty of each task, i.e. the inherent noise level of the data. The initial value is usually set to a small random number to ensure training stability.

[0067] This lays the foundation for subsequent dynamic weight balancing. By designing a differentiated loss function, it accurately matches the characteristics of each task. Furthermore, by introducing noise parameters, the model can adaptively perceive differences in task reliability, thus avoiding optimization bias caused by fixed weights.

[0068] S242. During the forward propagation process of each training batch, based on the current noise parameter value, the adaptive weight of the loss of each task is dynamically calculated according to the uncertainty weighting principle.

[0069] Specifically, during the forward propagation of each training batch, adaptive weights for the loss of each task are dynamically calculated based on the current noise parameter value according to the uncertainty weighting principle. This is specifically implemented by using the learnable noise parameter σ... t Calculate weight w t =1 / (2σ t ²), where σ t The logarithmic parameter, such as σ, is derived from the exponential operation. t =exp(logσ) t ).

[0070] The principle is based on the Bayesian modeling framework, which treats the task loss as following a Gaussian distribution, with the weights inversely proportional to the task uncertainty. That is, the greater the noise, the lower the weight, thereby automatically reducing the contribution of high-noise tasks to the total loss while increasing the impact of low-noise tasks. The technical effect of this step is to achieve dynamic normalization of the loss scale, solve the training imbalance problem caused by the difference in loss magnitude and uneven convergence speed among multiple tasks, and enable the model to adjust the optimization direction in real time at the batch level.

[0071] S243. The loss values ​​of each task are weighted and summed with the adaptive weights, and a regularization term is added to prevent the noise parameters from increasing indefinitely during training, so as to obtain the total loss value after fusion.

[0072] Specifically, the loss values ​​of each task are summed with the adaptive weights using a weighted average, and a regularization term is added to prevent noise parameters from increasing indefinitely during training. This process involves first calculating the weighted loss sum, such as L... total =Σ(w t ·L t Then, a regularization term is introduced and added to the weighted loss to form the final total loss.

[0073] The principle originates from the derivation of maximum likelihood estimation. The regularization term penalizes excessively large noise parameters, logσ t With σ tThe weights are increased to prevent the weights from approaching zero and causing the tasks to be ignored, thus ensuring that all tasks can participate in model optimization. The technical effect of this step is to construct a joint optimization objective with theoretical guarantees. It balances the contribution of multi-task loss through weighted summation and maintains training stability through regularization mechanism to prevent the divergence of noise parameters, ensuring that the model maintains the ability of each task to learn collaboratively during long-term training.

[0074] S244. During backpropagation, the gradient of the total loss value with respect to the model's master parameters and the noise parameters of each task is calculated. The optimizer is used to update the model's master parameters and the noise parameters simultaneously, so as to realize the training iteration of the multi-level classification deep learning model that automatically adjusts the weights according to the current learning status of each task.

[0075] Specifically, during backpropagation, the gradients of the total loss with respect to the model's principal parameters and the noise parameters of each task are calculated. The optimizer then simultaneously updates both the principal parameters and the noise parameters. The process involves calculating the partial derivatives of the total loss with respect to the model weights and noise parameters using the chain rule, and then simultaneously updating these two types of parameters using gradient descent. The principle is to treat the noise parameters as trainable variables, allowing the optimization process to not only adjust the model's feature extraction capabilities but also automatically learn the optimal weight allocation strategy for each task, forming a closed-loop feedback loop. For example, when the task loss is large, the gradient drives σ. t The adjustment, in turn, affects the weights; the technical effect of this step is to achieve fully end-to-end dynamic weight learning, which can automatically optimize the weight configuration based on the real-time training status without manual intervention, and ultimately improve the overall generalization performance and robustness of the model in multi-task scenarios.

[0076] S3. Based on preset multi-type callback functions, optimize and train the multi-level deep learning model, and input the real-time generated augmented dataset into the optimized multi-level classification deep learning model to output drilling and blasting parameter recommendations based on lithology identification results.

[0077] Specifically, callback functions are a set of functions executed at specific stages of the training process (such as after each epoch). They are used to implement various monitoring, control, and logging functions during training and are key tools for optimizing the training process and model performance. Three core callback functions are used during model training.

[0078] In the description of this invention, optimizing the training of a multi-level deep learning model based on preset multi-type callback functions includes: S31. Divide the training process of the multi-level classification deep learning model into several stages. Use early stopping to monitor the change of validation loss on the validation set at the end of each stage. If the validation loss does not improve within a continuous patience value, terminate the training and restore the best weight in the monitoring index.

[0079] Specifically, step S31 employs early stopping to prevent overfitting during model training. Early stopping monitors the change in val_loss (validation loss set) at the end of each epoch. If val_loss does not improve within a continuous patience value (patience=10), training automatically terminates. After stopping, restoring best_weights=True ensures that the weights are restored to the weights of the epoch with the best monitored metrics throughout the entire training process.

[0080] S32. Use model checkpoints to monitor the validation loss at each stage. When each stage ends, if the validation loss of the current stage is lower than the validation loss of all previous stages, the optimal multi-level classification deep learning model is saved to the specified file.

[0081] Specifically, step S32 employs ModelCheckpoint to ensure that after training, we obtain the model version with the strongest generalization ability, rather than a final version that may be overfitting. ModelCheckpoint also monitors val_loss. At the end of each epoch, if val_loss is lower than in all previous epochs, save_best_only=True and save the current best model to the specified file 'best_model.keras'.

[0082] S33. Use a logger to record the training loss, validation loss, training accuracy, and validation accuracy at each stage, automatically append them to the same CSV file, and plot the loss vs. accuracy curve.

[0083] Specifically, step S33 uses a CSV logger to record the training loss, validation loss, training accuracy, and validation accuracy for each epoch, automatically appending the data to a CSV file. The data saved to the CSV file can be read and used to plot loss and accuracy curves, providing a visual analysis of the model's learning process and convergence. Figure 5 .6.7) S34. Establish multi-task performance and engineering physical constraints, perceive the tasks and loss values ​​of multi-level deep learning models in real time for verification, and trigger loss correction based on the verification results.

[0084] In the description of this invention, establishing multi-task performance and engineering physical constraints, real-time sensing of the tasks and loss values ​​of multi-level deep learning models for verification, and triggering loss correction based on the verification results include: S341. Calculate the F1 score of the three branches of the multi-level task deep learning model on the validation set in real time. Calculate the absolute value of the difference between the F1 scores of any two tasks as a measure of the performance balance between tasks. Based on the preset performance difference threshold, if the absolute value of the difference between the F1 scores of any pair of tasks exceeds the performance difference threshold, a task imbalance warning is triggered.

[0085] S342. Upon receiving a task imbalance warning signal, automatically activate a targeted learning rate adjustment strategy to reduce the learning rate of the backbone feature extraction network shared by all tasks, and proportionally increase the learning rate of the branch network to which the imbalanced task belongs.

[0086] S343. Establish an engineering physics rule base, set the logical relationships between different construction methods, machinery subcategories, or rock types and drilling and blasting parameter recommendations, and input the model's prediction results into the engineering physics rule base for logical compliance verification during the training process.

[0087] Finally, it predicts a single image of the blast hole or a dataset of predicted images in real time; it outputs the lithology based on the construction method (mechanical method / drill-blast method). If it is a mechanical method, it outputs a subclass (TBM method / shield method); if it is a drill-blast method, it outputs the lithology (granite / limestone / sandstone). Furthermore, it outputs recommended drill-blast parameters based on the lithology identification results according to the recommend_blast_parameters function.

[0088] Secondly, the present invention also provides an intelligent recognition system for tunnel muck images based on multi-task deep learning, the system comprising: Image acquisition and processing module 1 is used to acquire images of the cavitation debris and shared data, construct the original dataset, and preprocess the original dataset using a dynamic adaptive data augmentation strategy and a focus loss function to obtain the augmented dataset; Model building and optimization module 2 is used to construct a multi-level classification deep learning model with a multi-branch output structure based on a preset backbone feature extraction network and pre-trained weights, output the probability distribution results of different classifications respectively, and set the loss function and loss weight of the multi-level classification deep learning model. Model training output module 3 is used to optimize and train a multi-level deep learning model based on preset multi-type callback functions, and input the real-time generated augmented dataset into the optimized multi-level classification deep learning model to output drilling and blasting parameter recommendations based on lithology identification results.

[0089] The following detailed description, in conjunction with specific embodiments, further illustrates the intelligent recognition method for tunnel muck images based on multi-task deep learning designed in this invention.

[0090] 1. Data processing and augmentation strategy algorithms First, the directory is traversed to obtain the paths of all image files. Then, data augmentation is performed using the dynamic adaptive enhancement strategy and focus loss function mentioned in the technical solution. The specific method is as follows: (1) Random horizontal flip: This operation mirrors the image along the vertical central axis with a 50% probability. Its mathematical expression is: ; in, For the original image, To perform a horizontal flip, W represents its width. These are image coordinates. Width direction, Height direction. Random horizontal flipping enhances the model's orientation invariance, making it independent of the specific orientation of the hole fragment image and simulating different shooting angles.

[0091] (2) Random brightness adjustment: This operation performs a global random offset on the intensity values ​​of all images, and its mathematical expression is: ; ; in, For the clipping function, To adjust brightness, ensure the range is within [0, 1]. For normalized images, These represent the offset factor and its range. Random brightness adjustment simulates changes in lighting conditions inside the tunnel, improving the model's robustness to lighting fluctuations.

[0092] (3) Random Contrast Adjustment: This operation adjusts the contrast by randomly scaling the offset of pixel values ​​relative to the image mean. Its mathematical expression is: ; ; in, For contrast, They represent contrast factors, It is the average of all image pixel values, where H, W, and C are the image height, width, and number of channels, respectively. These are the image height index, width index, and channel index, respectively. Random contrast adjustment makes the model focus more on texture and structure rather than absolute pixel values, improving the robustness of the model quality.

[0093] (4) Random Saturation Adjustment: This operation adjusts the color saturation channel in the HSV color space, mainly in the following three steps: a. The mathematical expression for converting an image from the RGB color space to the HSV color space is: ; b. Scaling the saturation channel (s), the mathematical expression for which is: ; c. Convert the image back to RGB space: ; ; in, For color space, These are the hue channel, saturation channel, and brightness channel, respectively. For random scaling factor, This is the adjusted saturation channel. This is the final output image after saturation adjustment. Through the above three steps, the model's overfitting to specific colors is reduced, and its robustness to color changes is enhanced.

[0094] (5) Random 90° rotation: This operation randomly rotates the image by 0°, 90°, 180°, or 270°. Its mathematical expression is: ; ; in, For rotation, This indicates the number of rotations (90°). exist Random values ​​are selected from the data. Random 90° rotations ensure that the model can be correctly identified regardless of how the cavity image is positioned.

[0095] (6) Normalization: All enhanced image pixel values ​​are scaled to a standard range, the mathematical expression of which is: ; in, This is for normalization. Its function is to map pixel values ​​from [0, 255] to [0, 1], accelerating convergence during model training and improving the stability of numerical calculations.

[0096] (7) Focus loss function: In this invention, it is mainly applied to the classification of construction methods. The formula used is: ; in, It is the focus loss function; It is the probability predicted by the model that the object belongs to the true category; It is a balancing factor used to adjust the weights in positive and negative samples; It is a modulation factor used to reduce the loss contribution of easily separable samples.

[0097] Then, the backbone network EfficientNet-B4 was constructed, and the Adam algorithm was used as the model optimizer. During training, early stopping functions were used to avoid overfitting, and model checkpoints were used to save the optimal model weights. The complete model execution process is as follows: Figure 3 .

[0098] Data source: The dataset of tunnel debris images used in this embodiment contains 284 original images, which are collected from actual tunnel engineering projects and the public database of the National Rock, Mineral and Fossil Resources Sharing Platform.

[0099] 2. System Execution and Results Data Preprocessing: The data loading and preprocessing module is initiated. The system identifies 55 original samples for TBM and 45 for shield tunneling, and a total of 126 samples for the drill-and-blast method, showing a significant imbalance. Therefore, the system executes a dynamic adaptive data augmentation strategy and a focus loss function: data augmentation is performed according to the method described in section 1 of the technical solution. The augmented images and the original images are merged and divided into training and validation sets in an 8:2 ratio.

[0100] Model Training: A multi-task classification model based on EfficientNet-B4 was constructed, and the loss function, weights, and optimizer were set as described in the invention. The maximum number of training epochs was set to 100, and the batch size was 32. An early stop callback was triggered during training, terminating training at epoch 31 (patience value 10), and automatically restoring the model weights with the lowest validation loss. The final model was saved as tunnel_muck_classifier.keras.

[0101] Model Evaluation: The best model was evaluated on the validation set, and the results are as follows: Total loss: 0.2055; Accuracy rate of construction method classification: 99.0%; Mechanical subclass classification accuracy: 99.0%; The accuracy rate of lithological classification using the drill-and-blast method is 99.0%.

[0102] Real-time prediction 1: To verify the system's practicality, an unknown image of tunnel embankment muck was selected ( Figure 8 The system performs predictions. The system execution flow is as follows: Image preprocessing: Read the image, decode it into RGB, scale it to 224x224 pixels, normalize the pixel values, and expand the batch dimension.

[0103] Model inference: Input the processed tensor into the loaded optimal model.

[0104] Result Analysis: The model outputs the construction method as "Mechanical Method" (confidence level 0.995). Activating the Mechanical Method sub-branch then outputs "Shield Tunneling Method" (confidence level 0.999). The lithology branch is not activated, and the output is "Not Applicable".

[0105] Real-time prediction 2: To verify the system's practicality, an unknown TBM-generated cavity image was selected for prediction. The system execution flow is as follows: Image preprocessing: Read the image, decode it into RGB, scale it to 224x224 pixels, normalize the pixel values, and expand the batch dimension.

[0106] Model inference: Input the processed tensor into the loaded optimal model.

[0107] Result Analysis: The model outputs the construction method as "Mechanical Method" (confidence level 0.999). Activating the Mechanical Method sub-branch then outputs "TBM" (confidence level 0.999). The lithology branch is not activated, and the output is "Not Applicable".

[0108] Real-time prediction 3: To verify the system's practicality, an unknown granite image was selected for prediction. The system execution flow is as follows: Image preprocessing: Read the image, decode it into RGB, scale it to 224x224 pixels, normalize the pixel values, and expand the batch dimension.

[0109] Model inference: Input the processed tensor into the loaded optimal model.

[0110] Result Analysis: The model outputs the construction method as "drill and blast" (confidence level 0.996). Activating the lithology branch then outputs "granite" (confidence level 0.999). Recommended drilling and blasting parameters include using high-power emulsion explosives, reducing hole spacing by 10%-15%, increasing charge by 15%, and applying sequential detonation technology. The mechanical subclass was not activated, resulting in an "Inapplicable" output.

[0111] Real-time prediction 4: To verify the system's practicality, an unknown limestone image was selected for prediction. The system execution flow is as follows: Image preprocessing: Read the image, decode it into RGB, scale it to 224x224 pixels, normalize the pixel values, and expand the batch dimension.

[0112] Model inference: Input the processed tensor into the loaded optimal model.

[0113] Result Analysis: The model outputs the construction method as "Drill and Blast" (confidence level 0.998). Activating the lithology branch then outputs "Limestone" (confidence level 0.999). Recommended drilling and blasting parameters are: using medium-powered ammonium nitrate oil explosives, maintaining standard hole spacing and charge amount, and using wedge-cut detonation. The mechanical sub-branch was not activated, outputting "Not Applicable".

[0114] Real-time prediction 5: To verify the system's practicality, an unknown sandstone image was selected for prediction. The system execution flow is as follows: Image preprocessing: Read the image, decode it into RGB, scale it to 224x224 pixels, normalize the pixel values, and expand the batch dimension.

[0115] Model inference: Input the processed tensor into the loaded optimal model.

[0116] Result Analysis: The model outputs the construction method as "drill and blast" (confidence level 0.998). Activating the lithology branch then outputs "sandstone" (confidence level 0.999). Recommended drilling and blasting parameters include using low-power explosives (or expanding agents), increasing hole spacing, reducing charge quantity, and employing pre-splitting blasting to effectively control vibration and avoid excessive fragmentation. The mechanical sub-branch was not activated, outputting "not applicable." Real-time prediction results are shown in Table 1.

[0117] Table 1: Summary Table of Real-time Prediction Results Conclusion: The implementation case successfully reproduced the entire process of this invention. The results show that the system described in this invention can efficiently and accurately perform hierarchical intelligent identification of tunnel muck based on "construction method → ​​mechanical subclass / lithology → drilling and blasting parameter recommendation," exhibiting particularly excellent performance in resolving sample imbalance and achieving high-precision identification and decision-making, fully meeting the real-time, automated muck classification requirements of construction sites.

[0118] In summary, by utilizing the above-mentioned technical solutions of this invention, the present invention possesses hierarchical recognition and decision-making capabilities. Using EfficientNet-B4 as the backbone feature extraction network, a hierarchical classification model with multi-branch outputs is constructed, enabling simultaneous identification of construction methods (mechanical or drill-and-blast methods) and specific lithology classification (granite, limestone, or sandstone). For the first time, it achieves conditional multi-level recognition based on "construction method - mechanical subclass / lithology - drill-and-blast parameter recommendation," with the recognition results directly guiding the adjustment of drill-and-blast construction parameters and improving the utilization rate of excavated material resources. This invention achieves data imbalance optimization by introducing a dynamic adaptive data augmentation strategy and a focus loss function, effectively alleviating the sample class imbalance problem and significantly improving the recognition performance of minority class samples. With the "dynamic adaptive data augmentation + focus loss" strategy, the optimal model achieves a construction method accuracy of 99.0% and a lithology classification accuracy of 99.0%, solving the problem of small sample class recognition. This invention achieves high accuracy and real-time performance, specifically based on shared feature extraction and multi-branch design using EfficientNet-B4. The optimal model achieves 99.0% accuracy in construction methods and 99.0% accuracy in lithology classification on the validation set. Inference time per image is rapid, supporting real-time decision-making at construction sites. This invention is also low-cost and easy to deploy, requiring only a regular camera to capture images. The model can be exported as a tunnel_muck_classifier.keras file and deployed on edge computing devices (such as industrial tablets), significantly reducing the cost and barrier to entry for engineering applications.

[0119] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

Claims

1. A method for intelligent recognition of tunnel muck images based on multi-task deep learning, characterized in that, include: S1. Obtain images of the hole debris and shared data, construct the original dataset, and preprocess the original dataset using a dynamic adaptive data augmentation strategy and a focus loss function to obtain the augmented dataset; S2. Based on the preset backbone feature extraction network and pre-trained weights, construct a multi-level classification deep learning model with a multi-branch output structure, output the probability distribution results of different classifications respectively, and set the loss function and loss weight of the multi-level classification deep learning model. S3. Based on preset multi-type callback functions, optimize and train the multi-level deep learning model, and input the real-time generated augmented dataset into the optimized multi-level classification deep learning model to output drilling and blasting parameter recommendations based on lithology identification results.

2. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 1, characterized in that, The process involves acquiring cavity debris images and shared data to construct an original dataset. The original dataset is then preprocessed using a dynamic adaptive data augmentation strategy and a focus loss function to obtain an augmented dataset, which includes: S11. Obtain images of the slag generated during tunnel construction, and combine them with the shared data provided by the rock, mineral and fossil resource sharing library to construct a three-level layered original dataset. The three-level layered structure includes construction methods, mechanical subcategories or lithological classifications, and recommended drilling and blasting parameters. S12. Traverse the directory to obtain all image file paths and generate a three-level one-hot encoded label vector; S13. A dynamic adaptive data augmentation strategy based on the target sample size is used to enhance the images of slag from different types of construction methods, generating diverse sample images. S14. Based on a preset focus loss function, train the sample images corresponding to construction methods with an absolute number greater than a preset threshold to generate trained and adjusted sample images; then merge the enhanced sample images with the original cavity images to form the final augmented dataset. S15. Divide the augmented dataset into training and validation sets according to a preset ratio, and build a high-efficiency data pipeline based on an efficient data input pipeline building tool to support parallel decoding, caching, and prefetch optimization.

3. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 2, characterized in that, The dynamic adaptive data augmentation strategy based on the target sample size enhances images of caving debris from different types of construction methods, generating diverse sample images including: S131. Calculate the feature map of each cave debris image by perceiving the visual characteristics of the image, extract the texture complexity features and key area saliency map of the cave debris image, and combine the preset difficulty coefficient corresponding to the construction method described in the cave debris image to construct a comprehensive dynamic enhanced difficulty score. S132. Construct a lightweight policy network, using the losses of the original samples, the most augmented samples, and the non-augmented samples in the current training cycle as state inputs, and combining the augmentation difficulty score, dynamically output the augmentation operation magnitude customized for each training sample through the superior actor-critic algorithm. S133. Based on the enhancement operation amplitude, the enhancement operation is performed cyclically on each cavity debris image, and the cavity debris images at the end of the cycle are integrated as diverse sample images. The random change operation includes random horizontal flipping, random brightness adjustment, random contrast adjustment, random saturation adjustment, random 90° rotation, image scaling to a uniform size, and pixel value normalization.

4. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 3, characterized in that, The formula for calculating the enhanced difficulty score is as follows: ; ； In the formula, Indicates the difficulty score; indicates the weighting coefficients of different factors; represents the significance plot. S The global average; The preset difficulty coefficient indicates the construction method type to which the caving debris image belongs; The texture complexity score represents the image of the hole debris. N This represents the total number of local blocks in the cavity image; The first one in the image of the hole debris i A local area; This represents the function for calculating the standard deviation of a given image patch.

5. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 3, characterized in that, The lightweight policy network is constructed using the losses of the original samples, the most augmented samples, and the non-augmented samples in the current training cycle as state inputs. Combined with an augmentation difficulty score, it dynamically outputs the augmentation operation magnitude customized for each training sample through the dominant actor-critic algorithm, including: S1321. Set the training period, collect the historical loss values ​​fed back by the multi-level classification deep learning model to construct the state vector, and concatenate the state vector with the enhancement difficulty score of the hole debris image to merge it into the input state of the policy network. S1322. Construct a lightweight neural network as a policy network, and introduce an activation function in the output layer to generate continuous enhancement magnitude values ​​between 0 and 1 for each available image enhancement operation. S1323. Construct a value network to evaluate the current state, use a policy network to output the augmentation magnitude according to the current state, and calculate the reward signal based on the performance feedback of the target model after training on the augmented sample images. S1324. Use the dominance function to guide the parameter update direction of the policy network. Calculate the difference between the reward obtained by taking an actual action in a specific state and the baseline value of that state predicted by the value network through the dominance function, and dynamically output the magnitude of each enhancement operation.

6. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 1, characterized in that, The multi-level classification deep learning model, based on a preset backbone feature extraction network and pre-trained weights, constructs a multi-branch output structure, outputting probability distribution results for different classifications. The loss function and loss weights of the multi-level classification deep learning model are defined as follows: S21. A 128-dimensional fully connected layer is connected, and a non-linear feature mapping is introduced using the ReLU activation function. A random deactivation layer and an output layer with three neurons are also introduced to output the probability distribution of different types of construction methods, forming the first branch output structure of the multi-level classification deep learning model. S22. A 128-dimensional fully connected layer and a random deactivation layer are connected, and an output layer with two neurons is introduced to output the probability distribution of different types of mechanical subclasses, forming the second branch output structure of the multi-level classification deep learning model. S23. Connect a 128-dimensional fully connected layer and a random deactivation layer, and introduce an output layer with three neurons to output the probability distribution representing different types of lithology, forming the second branch output structure of the multi-level classification deep learning model. S24. A dynamic loss weight balancing strategy is adopted to construct a multi-task loss function, and the weights are automatically adjusted according to the uncertainty of multi-tasks to achieve adaptive optimization of the loss function.

7. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 6, characterized in that, The method of constructing a multi-task loss function using a dynamic loss weight balancing strategy and automatically adjusting the weights based on the uncertainty of the multi-task includes: S241. Set task-specific loss functions for the output structures of different branches of the multi-level classification deep learning model, and initialize the associated learnable noise parameters. The first branch output structure adopts the focus loss function, and the second and third branch output structures adopt the cross-entropy loss function. S242. During the forward propagation process of each training batch, based on the current noise parameter value, the adaptive weight of the loss of each task is dynamically calculated according to the uncertainty weighting principle. S243. The loss values ​​of each task are weighted and summed with the adaptive weights, and a regularization term is added to prevent the noise parameters from increasing indefinitely during training, so as to obtain the total loss value after fusion. S244. During backpropagation, the gradient of the total loss value with respect to the model's master parameters and the noise parameters of each task is calculated. The optimizer is used to update the model's master parameters and the noise parameters simultaneously, so as to realize the training iteration of the multi-level classification deep learning model that automatically adjusts the weights according to the current learning status of each task.

8. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 1, characterized in that, The optimization training of the multi-level deep learning model based on preset multi-type callback functions includes: S31. Divide the training process of the multi-level classification deep learning model into several stages. Use early stopping to monitor the change of validation loss on the validation set at the end of each stage. If the validation loss does not improve within a continuous patience value, terminate the training and restore the best weight in the monitoring index. S32. Use model checkpoints to monitor the validation loss of each stage. When each stage ends, if the validation loss of the current stage is lower than the validation loss of all previous stages, the optimal multi-level classification deep learning model is saved to the specified file. S33. Use a logger to record the training loss, validation loss, training accuracy, and validation accuracy at each stage, automatically append them to the same CSV file, and plot the loss vs. accuracy curve. S34. Establish multi-task performance and engineering physical constraints, perceive the tasks and loss values ​​of multi-level deep learning models in real time for verification, and trigger loss correction based on the verification results.

9. The intelligent recognition method for tunnel muck images based on multi-task deep learning according to claim 8, characterized in that, The process of establishing multi-task performance and engineering physical constraints, real-time sensing of the tasks and loss values ​​of multi-level deep learning models for verification, and triggering loss correction based on the verification results includes: S341. Real-time calculation of the F1 score of the output structure of the three branches on the validation set of the multi-level task deep learning model. Calculate the absolute value of the difference between the F1 scores of any two tasks as a measure of the performance balance between tasks. Based on the preset performance difference threshold, if the absolute value of the difference between the F1 scores of any pair of tasks exceeds the performance difference threshold, a task imbalance warning is triggered. S342. Upon receiving a task imbalance warning signal, automatically activate a targeted learning rate adjustment strategy to reduce the learning rate of the backbone feature extraction network shared by all tasks and proportionally increase the learning rate of the branch network to which the imbalanced task belongs. S343. Establish an engineering physics rule base, set the logical relationships between different construction methods, machinery subcategories, or rock types and drilling and blasting parameter recommendations, and input the model's prediction results into the engineering physics rule base for logical compliance verification during the training process.

10. A tunnel muck image intelligent recognition system based on multi-task deep learning, used to implement the tunnel muck image intelligent recognition method based on multi-task deep learning as described in any one of claims 1-9, characterized in that, The system includes: The image acquisition and processing module is used to acquire images of the cavitation debris and shared data, construct the original dataset, and preprocess the original dataset using a dynamic adaptive data augmentation strategy and a focus loss function to obtain the augmented dataset. The model building and optimization module is used to construct a multi-level classification deep learning model with a multi-branch output structure based on a preset backbone feature extraction network and pre-trained weights, output the probability distribution results of different categories respectively, and set the loss function and loss weight of the multi-level classification deep learning model. The model training output module is used to optimize and train a multi-level deep learning model based on preset multi-type callback functions, and input the real-time generated augmented dataset into the optimized multi-level classification deep learning model to output drilling and blasting parameter recommendations based on lithology identification results.

Images