Multi-element visual identification method for underground cavern dark dust scene

Abstract

The invention provides a multi-element visual identification method for an underground cavern dark dust scene, and relates to the field of tunnel construction intelligent management. The method comprises the following steps: acquiring image data in monitoring video data of an underground cavern dark dust scene; filtering the image data, and performing multi-element classification labeling through LabImg software to generate an image data set; in the YOLOv5 target detection model, a CBAMC3 module is used for replacing a C3 module in the Backbone; inputting the image data set into a target detection model for iterative training to a preset round; and accessing the trained target detection model to a monitoring video source of an underground cavern dark dust scene, and carrying out multi-element real-time visual identification. According to the method, more efficient and accurate real-time visual identification monitoring can be realized in the underground cavern under the conditions of man-machine shielding, different directions and small target identification scenes.

Description

Technical Field

[0001] The present invention relates to the field of intelligent management of tunnel construction, and in particular to a multi-factor visual recognition method for dark dust scenes in underground caverns. Background Art

[0002] Among many projects, tunnel projects are the most important in terms of the number of constructions, the difficulty of construction, and the overall construction period. Underground caverns are usually classified as underground secret projects. They generally show the characteristics of large burial depth and large span of underground engineering construction. Their working scenes are limited in space, the work process is frequently switched, and the interaction with people is also very frequent. In the fast and continuous construction process, various processes are intertwined and interfered with each other, making the construction site present the characteristics of high mobility, high risk, a large number of operations, dim light, and a lot of dust. Compared with ground projects, the risk factors faced by underground caverns and other underground projects during construction are more complex and difficult to detect.

[0003] During the construction of underground caverns, traditional construction progress, quality assessment and identification of safety hazards mainly rely on manual inspections. This method has limited efficiency, is difficult to achieve exhaustive inspections of all scenarios, and has a high rate of misjudgment. The tunnel is dimly lit, has a high dust concentration, and is continuously advancing multiple steps such as excavation, support and secondary lining, making it difficult to achieve efficient monitoring in all time and space, from multiple angles and in a standardized manner.

[0004] On the other hand, the existing technology applies a variety of target detection algorithms to the management process of underground cavern construction sites. Among them, the two-stage algorithms represented by Region-CNN (Region-Convolutional Neural Networks), Faster R-CNN (Faster Region-Convolutional Neural Networks) and Mask R-CNN (Mask Region-Convolutional Neural Networks) have significantly improved accuracy, but the running speed is average, which is difficult to meet the real-time monitoring needs of underground cavern construction sites. In addition, although the traditional YOLO (You Only Look Once) algorithm can achieve end-to-end detection and has a fast detection speed, the detection accuracy is low. The YOLO algorithm does not use regional sampling technology, which limits the number and angle of monitoring at the underground cavern construction site. In addition, monitoring needs to be continuously advanced with the construction progress of the face, resulting in technical problems such as occlusion, overlap and inconsistent direction of the monitored personnel and mechanical equipment during imaging, making it difficult to provide high-quality image information. Summary of the invention

[0005] In order to solve the above-mentioned technical problems existing in the prior art, the present invention aims to provide a visual recognition method which can solve the problems of dim, dusty and overlapping occlusions in the monitoring screen of the underground cavern construction site to a certain extent, and accurately identify various construction elements in the real-time scene of the underground cavern construction on the monitoring background.

[0006] To achieve the above technical objectives, the present invention provides a multi-factor visual recognition method for underground cavern dark dust scenes, and the technical solution includes the following steps:

[0007] Step S1: collecting image data from monitoring video data of a dark dust scene in an underground cave;

[0008] Step S2: Filter the image data and perform multi-factor classification and annotation through LabImg software to generate an image data set;

[0009] Step S3: constructing a target detection model;

[0010] In the YOLOv5 target detection model, the CBAMC3 module (Convolutional BlockAttentionModule Concentrated-Comprehensive ConvolutionBlock) is used to replace the C3 module (Concentrated-ComprehensiveConvolutionBlock) in Backbone;

[0011] Step S4: input the image data set into the target detection model for iterative training to a preset number of rounds;

[0012] Step S5: Connect the trained target detection model to the surveillance video source of the underground cavern dark dust scene to perform real-time visual recognition of multiple factors.

[0013] Preferably, the types of multi-element classification labeling include construction workers, rock drilling trolleys, wet shotcrete trolleys, loaders, dump trucks, lining trolleys, steel bar trolleys, maintenance trolleys, invert trolleys, electric welders, vibrators, wires, ladders and scaffolding.

[0014] Furthermore, after step S1, the following steps are further included:

[0015] The image data x 1 Import the self-calibration lighting module to perform self-calibration lighting processing.

[0016]

[0017] In the formula, t is the stage number, t≥1, x tis the input light intensity at stage t, G(·) represents self-calibration, v t The output light intensity after self-calibration in stage t, y is the self-calibration parameter, s t 、z t is the intermediate calculation amount, u t is the residual term of stage t, θ is the weight, K θ , H θ In order to introduce parameterized operators whose parameters can be learned, represents mapping, and F(·) represents lighting conversion;

[0018] After T stages of self-calibration illumination processing, the output image data x T+1 ; Wherein, T is the total number of stages of the self-calibration lighting module.

[0019] Preferably, the total loss L during the self-calibration illumination process is calculated total ,

[0020] L total =αL f +βL s ;

[0021] In the formula, α and β are positive balance parameters, L f For fidelity, L s To smooth the loss;

[0022]

[0023] Where N is the total number of pixels, i is the pixel ordinal number, A(i) represents the 5×5 pixels adjacent to the i-th pixel, and w i,j is the weight coefficient, is the imaging data of the i-th pixel;

[0024]

[0025] Where c is the image channel in the YUV color space, y i,c is the self-calibration parameter of the ith pixel in image channel c, is the intermediate calculation amount of the i-th pixel in the image channel c at stage t-1, and the standard deviation of the Gaussian kernel;

[0026] By minimizing the total loss L total Determine the optimal weight coefficient w i,j .

[0027] Preferably, constructing the target detection model also includes replacing the concat module in Neck with a Bidirectional Feature Pyramid Network (BiFPN) module.

[0028] Preferably, step S4 comprises:

[0029] Step S41: input the image data set into the Backbone module;

[0030] Step S411: In the Conv module, convolution, normalization and nonlinear processing are performed on each image data in sequence through the convolution layer, BN layer and activation function layer;

[0031] Step S412: Input the nonlinearly processed image data F into the CBAMC3 module.

[0032]

[0033] Where M C (F) and M S (F) are the channel attention output feature map and the spatial attention output feature map, σ(·) represents the Sigmoid function, They are the average pooling features of the channel attention mechanism and the spatial attention mechanism, respectively. They are the maximum pooling features of the channel attention mechanism and the spatial attention mechanism, respectively. 7×7 Represents a 7×7 convolution operation;

[0034] Step S413: Output the spatial attention feature map M S (F) Input SPPF module;

[0035] Concatenate all spatial attention output feature maps M S (F) is then input into the fully connected layer for dimensionality reduction to obtain the feature vector

[0036] Step S42: Transform the feature vector Input Neck module for scale conversion,

[0037]

[0038] In the formula, is a feature vector of scale k×k, 3≤k≤7, Conv(·) represents convolution, and Resize(·) represents upsampling or downsampling;

[0039] Step S43: inputting the scale-converted feature vector into the Head module;

[0040] Set the target box through Anchors, and construct the corresponding detection box through K-means clustering method;

[0041] Classify the detection box by combining the fully connected layer and the Softmax function through Classification;

[0042] Use the fully connected layer through regression to regress the position and size of each detection box;

[0043] Step S44: Set training parameters and start training until the preset number of rounds is completed.

[0044] Preferably, the maximum area enclosed by the PR curve (Precision-Recall Curve) and the coordinate axis is calculated, and the corresponding weight is taken as the optimal weight. The area enclosed by the PR curve and the coordinate axis is the mAP (meanAverage Precision) value.

[0045] Preferably, setting the training parameters includes: setting the number of image data participating in the training in a single batch to 16, the momentum of the SGD (Stochastic Gradient Descent) optimizer to 0.937, the attenuation coefficient of the weight to 0.0005, the learning rate to 0.01, and the number of iterations to 100.

[0046] The technical solution provided by the present invention can realize more efficient and accurate real-time visual recognition monitoring in underground caves under human-machine occlusion, different directions, and small target recognition scenarios through the CBAMC3 module and the BiFPNM module. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is the learning framework of the self-calibrated lighting in the present invention.

[0048] Figure 2 This is a network structure diagram of the multi-factor visual recognition algorithm for underground cavern dark dust scenes in the present invention.

[0049] Figure 3 This is the structural diagram of the CBAMC3 module in the present invention.

[0050] Figure 4 This is a diagram of the pyramid network feature fusion structure in the present invention.

[0051] Figure 5 It is a flow chart of the multi-factor visual recognition method for dark dust scenes in underground caves in the present invention. DETAILED DESCRIPTION

[0052] Hereinafter, the technical solution provided by the present invention will be further elaborated in detail with reference to the accompanying drawings.

[0053] Step 1:

[0054] (1) Screening out specific construction elements of underground cavern construction scenes. The present invention combines the "SL400-2016 Safety Technical Regulations for the Installation of Mechanical and Electrical Equipment of Water Conservancy and Hydropower Projects", "SL 401-2007 Safety Technical Regulations for Construction Workers of Water Conservancy and Hydropower Projects" and the construction conditions of the exit section of the Yangtze-Hanjiang Water Diversion Tunnel of a specific project, and screens out 13 types of man-machine-environment construction elements, including construction personnel, rock drilling trolleys, wet spraying trolleys, loaders, dump trucks, lining trolleys (platforms), steel bar trolleys (platforms), maintenance trolleys, inverted trestles, electric welders, vibrators, wires, ladders, scaffolding, etc. Collect video data of water diversion tunnel construction scenes.

[0055] (2) Figure 1 As shown in the figure, the 13 types of construction element targets collected by the above screening are imported into the SCI (Self-Calibrated Illumination) learning framework. According to the Retinex retinal cortex theory, there is a connection between the weak light observation y and the expected clear image z.

[0056]

[0057] Where x represents the illumination component.

[0058] Generally speaking, illumination is regarded as the core component that needs to be optimized for low-light image enhancement. According to the Retinex theory, the estimated illumination is removed to further obtain the enhanced output. By introducing a mapping H with parameter C θ To learn about lighting, we provide a progressive perspective to model the task, with the basic units being:

[0059]

[0060] In the formula, u t and x t represents the residual term and illumination at stage t, t = 1., 2, ..., T-1, and is the total number of stages. It should be noted that due to the weight sharing mechanism, there is no θ The stage numbers are marked in , that is, the same architecture H and weights θ are used in each stage.

[0061] Parameterized operator H θ A simple residual representation is learned between illumination and low-light observations. In most areas, illumination and low-light observations are similar or linearly connected. Compared with the traditional method of learning residual representations by directly mapping low-light observations to illumination, the computational difficulty is greatly reduced. In particular, the performance is guaranteed and the stability is improved in terms of exposure control.

[0062] The above established process and the given training loss and data are used to obtain the enhanced model. But the cascade mechanism with multiple weight-sharing blocks will inevitably increase the foreseeable inference cost. The ideal situation is that the first block can output the expected result and meet the task requirements. At the same time, the results output by the subsequent blocks are similar to or even exactly the same as the first block. First, a module is defined to make the results of each stage converge to the same state. In order to bridge the input of each stage (except the first stage) with the low-light observation (that is, the input of the first stage), the convergence behavior between each stage is explored. A self-calibration module is introduced and added to the low-light observation to represent the difference between the input of each stage and the input of the first stage.

[0063] Specifically, the self-calibration module can be expressed as:

[0064]

[0065] In the formula, t≥1,v t s is the input after conversion at stage t, K θ To introduce parameterized operators whose parameters can be learned.

[0066] The conversion of the t-th level basic unit is:

[0067] F(x t )=F(G(x t ));

[0068] The self-calibration module constructed above integrates physical principles to gradually correct the input of each stage, indirectly affecting the output of each stage.

[0069] Taking into account the inaccuracy of the existing paired data, unsupervised learning is used to expand the capacity of the network, and the total loss is defined as:

[0070] L total =αL f +βL s ;

[0071] In the formula, α and β are positive balance parameters, L f For fidelity, L s To smooth the loss.

[0072] The fidelity loss is to ensure the consistency of the estimated illumination intensity with the input pixel level at each stage, expressed as:

[0073]

[0074] In fact, the function uses the redefined input y+s t-1 Constrain the output light intensity x t , rather than manually crafted data annotation input or ordinary low-light input.

[0075] In this task, the smoothness of the illumination is constant by default. The present invention adopts a smoothness term with a spatial variation norm, expressed as:

[0076]

[0077] Where N is the total number of pixels, i is the pixel ordinal number, A(i) represents the 5×5 pixels adjacent to the i-th pixel, and w i,j is the weight coefficient, is the imaging data of the i-th pixel; correspondingly, Imaging data of a neighboring pixel of the i-th pixel;

[0078]

[0079] Where c is the image channel in the YUV color space, y i,c is the self-calibration parameter of the ith pixel in image channel c, is the intermediate calculation amount of the i-th pixel in the image channel c at stage t-1, and the standard deviation of the Gaussian kernel. Correspondingly, y j,c is the self-calibration parameter of a neighboring pixel of the ith pixel in image channel c, It is the amount of calculation between a neighboring pixel of the i-th pixel in the stage t-1 in the image channel c.

[0080] In essence, the self-calibration module plays an auxiliary role to learn a better basic block (the illumination estimation block in the present invention), which is cascaded to generate the overall illumination learning process through a weight sharing mechanism. More importantly, the self-calibration module provides convergence between the results of each stage.

[0081] Step 2: After exporting the image set output by the self-calibration lighting framework in step 1, select 13 types of construction elements that need to be identified, including construction personnel, rock drilling trolleys, wet spraying trolleys, loaders, dump trucks, lining trolleys (platforms), steel trolleys (platforms), maintenance trolleys, arch trestles, electric welders, vibrators, wires, ladders, scaffolding, etc., according to the project situation, and use LabImg software to classify and annotate the effective images to produce a VOC2007 format data set. After annotation, it is divided into a training set and a validation set at a ratio of 8:2, and a txt format copy is saved for backup.

[0082] Step 3: If Figure 2As shown in the figure, after completing the self-calibration lighting framework of step 1 and the classification labeling of step 2, the collected data set is input into the network structure diagram of the construction multi-factor intelligent recognition algorithm of the dark dust environment of the underground cavern. The network structure Yolov5 diagram mainly consists of three parts: Backbone module, Neck module and Head module. In the architecture of Yolov5, CSPDarknet53 (Cross Stage Partial Darknet53) and ResNet (Residual Neural Network) are selected as its core network. The Backbone backbone network is used for image feature extraction. Its core role in the target detection task is to convert the initial input image into a multi-level feature map. The two network designs are relatively lightweight. On the basis of ensuring a high degree of detection accuracy, efforts are made to reduce the pressure of calculation and memory usage, thereby meeting the real-time recognition requirements of hydraulic tunnel elements. The core network architecture is mainly composed of Conv module, C3 module and SPPF module (Spatial Pyramid Pooling Fast). The Conv module is a core module frequently used in convolutional neural networks. It consists of a convolution layer, a BN layer (Batch Normalization) and an activation function layer. The C3 module enhances the extraction efficiency of key features of the construction site by expanding the depth and perception range of the network. In order to more effectively extract features and meet the needs of target recognition and positioning, the present invention replaces the C3 module with the CBAMC3 module during the algorithm construction process, thereby further enhancing the model's ability to extract multi-factor features in underground cavern construction scenes. The working principle of the SPPF module is to first perform pooling processing of various sizes on the input feature map to obtain a series of feature maps of different sizes. Next, these feature maps are connected in series, and dimensionality reduction processing is performed on the fully connected layer, and finally a feature vector of fixed size is obtained. The cooperation between modules can effectively capture key features in the construction site image, such as construction personnel and construction machinery, thereby improving the performance of target detection. Among them, Upsample and upsampling modules.

[0083] In view of the fact that the location of monitoring layout is limited during tunnel construction, and the layout point needs to move forward continuously with the excavation of the tunnel, there are common problems such as occlusion, different directions and sizes between people, machines and various construction elements in the monitoring image, which leads to reduced accuracy of target recognition. Therefore, it is necessary to process targets of different scales and sizes by improving the feature pyramid in the Neck module. Feature pyramid technology is a method for processing multi-scale target detection, which is achieved by adding feature layers of various scales to the core network. In the process of model construction, the present invention introduces a concise and efficient weighted feature pyramid network BiFPN module, whose core function is to integrate features from various resolutions, and further optimize the performance of the model by combining learnable weights with the fusion of multi-scale features. By introducing the BiFPN module, the model can more effectively adapt to the input characteristics of various scales and resolutions, and meet the mechanical mobility requirements of the construction site. This not only enhances the model's perception of multi-scale personnel and mechanical information in the water transfer tunnel construction scene, but also further improves the accuracy and robustness of model detection.

[0084] Head target detection, as the core component of feature pyramid target detection, covers multiple parts such as convolutional layers, pooling layers, and fully connected layers. It successfully implements multi-scale target recognition by extracting feature maps from the backbone network in the YOLOv5 model. This module is mainly divided into three parts: Anchors module, Classification module, and Regression module. The function of the Anchors module is to define target boxes with different sizes and aspect ratios, and create corresponding detection boxes through the K-means clustering method.

[0085] The K-means clustering method is used to construct the detection frame. In yolov5, the default anchor frame of Anchors is not used alone. Before starting training, the annotation information in the dataset will be checked, and the best recall rate of the annotation information of this dataset for the default anchor frame will be calculated. If it is less than 0.98, the K-means clustering method is needed to iteratively calculate the anchor frame that conforms to this dataset.

[0086] The anchor module sets the target frame. When setting the target frame in the target detection task, the anchor module parameters have three rows, each row has 9 values; and each row represents the application of different feature maps. Generally, it is hoped to detect small targets on large feature maps, because large feature maps contain more small target information. Therefore, the anchor values ​​on large feature maps are usually set to small values, while the values ​​on small feature maps are set to large values ​​to detect large targets. The Classification module uses a combination of a fully connected layer and a Softmax function to classify each detection frame to determine whether it covers the target object.

[0087] The Regression module uses a fully connected layer to achieve accurate regression of the position and size of each detection box.

[0088] The close cooperation of the above three parts ensures that the model can effectively complete the task of target detection at multiple scales. In order to further improve the accuracy of detection, YOLOv5 integrates GIoU loss (Generalized Intersection over Union loss), Mish activation function and multi-scale training method.

[0089] GIoU loss: The loss function of the target detection task consists of two parts: Classification Loss and Bounding Box Regression Loss. GIoU loss is used to calculate Bounding Box Regression Loss, which is a loss function calculation method in the target detection process. When the prediction box and the target box do not intersect, IoU(A,B) = 0, it cannot reflect the distance between A and B. At this time, the loss function is not differentiable, and IoULoss (Intersection over Union Loss) cannot optimize the situation where the two boxes do not intersect. GIoU considers the size of the minimum closure of the two rectangles (the minimum circumscribed rectangle of the two rectangles) based on IoU.

[0090] Mish activation function: The activation function introduces nonlinear factors to neurons, allowing the neural network to arbitrarily approximate any nonlinear function. It acts between the output and input of neuron nodes. The Mish activation function has the characteristics of no upper bound, lower bound, smoothness, and non-monotonicity. The effect of the Mish activation function on deep models is better than that of the ReLU activation function. No upper bound can avoid function saturation caused by excessive activation values.

[0091] From a macro perspective, the Head module object detection head plays a vital role in the model. By integrating the various functions of these components, the model can more accurately identify construction workers, construction equipment, and on-site environmental factors in the image.

[0092] The above is an introduction to the overall structure of the multi-factor intelligent visual recognition algorithm for construction in dark dusty environments in underground caverns. The specific improvements to the Backbone module and the Neck module are as follows.

[0093] Backbone attention mechanism optimization:

[0094] like Figure 3As shown in the figure, in underground cavern construction, due to its high mobility, complex environment, and a large number of operating equipment and personnel, it is difficult for the target detector to continuously lock on to key construction targets. Therefore, the CBAM module is introduced to focus on the correlation information of multiple channels, integrate the channel attention mechanism and the spatial attention mechanism, and embed these mechanisms into the existing model to achieve real-time recognition of on-site elements and the model fusion embedding structure.

[0095] Before calculating the attention of the channel, pooling helps to reduce the size of the feature map and the spatial dimension of the input data while retaining the main feature information, thereby reducing the number of parameters and computational burden in the model. When dealing with the maximum pooling problem, the maximum value in the local area is selected as the output, thereby retaining the most significant and important features of the on-site construction image. The average pooling method reveals the global characteristics of the local area by calculating the average value. When these two methods are applied at the same time, the uniqueness and integrity of the image channel can be taken into account when extracting the implicit association information, thereby ensuring that the final feature map has a closer connection with the actual construction image. The specific expression is as follows:

[0096]

[0097] Where M C (F) is the channel attention output feature map, σ(·) represents the Sigmoid function, w1 and w0 are weight coefficients, is the average pooling feature of the channel attention mechanism, It is the maximum pooling feature of the channel attention mechanism.

[0098] As a supplement to the channel attention mechanism, the spatial attention mechanism performs another maximum pooling and average pooling process on the feature map output by the channel attention mechanism, and finally performs convolution integration on these results to generate the spatial attention feature map M S (F),

[0099]

[0100] Where M S (F) is the spatial attention output feature map, is the average pooling feature of the spatial attention mechanism, is the maximum pooling feature of the spatial attention mechanism, f 7×7 Represents a 7×7 convolution operation;

[0101] Neck weighted bidirectional feature fusion:

[0102] Given that the location of monitoring arrangements during tunnel construction is limited, and the deployment points need to move forward as the tunnel is excavated, there are common problems such as occlusion, different directions and sizes between the construction elements of man-machine-environment in the monitoring image, which leads to reduced accuracy of target recognition. Therefore, the feature pyramid in the Neck module is used to process targets of different scales and sizes. The traditional FPN (Feature Pyramid Network) network does not take into account the difference in resolution when inputting features, which leads to unequal contributions to the fusion output, making target recognition more difficult. In order to reduce the impact of occlusion overlap and other problems, in addition to increasing the number of samples, BiFPN feature fusion is also performed. Different weights are set according to the priority of different input features, and its structural enhancement feature fusion can be used multiple times at the same time.

[0103] The specific implementation steps are as follows: Figure 4 As shown in Figure 1, four feature transmission modes from bottom to top and from top to bottom in the network are introduced. Specifically, Figure 4 (a), (b) and (c) are the corresponding feature transmission methods in FPN (Feature Pyramid Network), PANet (Path Aggregation Network) and NAS-FPN (Neural Architecture Search Feature Pyramid Network). The two-way information transmission allows information to flow bidirectionally at different levels, which helps to more comprehensively integrate high-level and low-level features, thereby strengthening the connection between contextual semantics. Figure 4 As shown in (d), the blue arrow part represents a path from top to bottom, which is used to convey the meaning of high-level features; the red arrow part is a channel from bottom to top, which is used to convey the location information of low-level features; the purple area adopts a cross-scale connection method, and by adding skip connections and bidirectional paths, the effect of weighted fusion and bidirectional cross-scale connection is achieved. In the figure, repeated blocks are repeated blocks.

[0104] Detailed steps:

[0105]

[0106] In the formula, is a feature vector of scale k×k, 3≤k≤7, Conv(·) represents convolution, and Resize(·) represents upsampling or downsampling operation. After finding a suitable conversion scale through scale conversion, the newly aggregated features are output

[0107] Step 4: As shown in Table 1, set the model training parameters and start training.

[0108] Table 1 Hyperparameter settings

[0109] Parameter name Numeric batch-size 16 momentum 0.937 decay 0.0005 learningrate 0.01 epochs 100

[0110] In the table, batch-size represents the number of images trained simultaneously in each batch, which is determined based on the GPU (Graphics Processing Unit) configuration of the CNC center at the construction site; the optimizer in the network model is SGD (Stochastic Gradient Descent), which has a fast descent rate but insufficient stability, while momentum represents momentum, which reflects the magnitude of change of the network in each iteration. Its main function is to speed up convergence and ensure the stability of the results, so that the graded descent reaches the optimal rate; decay represents the attenuation coefficient of the weight; learning rate represents the initial learning rate; epochs represents the number of iterations.

[0111] like Figure 5 As shown in the figure, after 100 iterations, the best weight is selected by combining the PR curve and the mAP value. This is the recognition algorithm in the multi-factor intelligent visual recognition technology for the construction of underground caverns in dark dusty environments of this project. The algorithm is connected with the monitoring video source to complete the entire process of intelligent real-time recognition of multi-factors in the construction of underground caverns in dark dusty environments.

[0112] In summary, the technical solution provided by the present invention can achieve more efficient and accurate real-time visual recognition monitoring in underground caves under human-machine occlusion, different directions, and small target recognition scenarios through the CBAMC3 module and BiFPN.

[0113] In addition to achieving the above-mentioned beneficial effects, each preferred scheme has also achieved the following beneficial effects: the self-calibration lighting learning framework can improve the recognition accuracy based on the dark and dusty characteristics of underground cavern construction scenes; the CBAMC3 module can further expand the depth and perception range of the network, and enhance the ability of the target detection model to extract multi-factor features in dark and dusty scenes in underground caverns; the BiFPN module can more effectively adapt to input characteristics of various scales and resolutions, and improve the mechanical mobility of the construction site; determining the optimal weight coefficient of the self-calibration lighting module can further improve the recognition accuracy; setting appropriate hyperparameters of the target detection model is conducive to improving the efficiency and accuracy of real-time visual recognition.

Claims

1. A multi-factor visual recognition method for underground cavern dark dust scenes, characterized in that: The following steps are involved: Step S1: collecting image data from monitoring video data of a dark dust scene in an underground cave; Step S2: Filter the image data and perform multi-factor classification and annotation through LabImg software to generate an image data set; Step S3: constructing a target detection model; In the YOLOv5 target detection model, the CBAMC3 module is used to replace the C3 module in Backbone; Step S4: input the image data set into the target detection model for iterative training to a preset number of rounds; Step S5: Connect the trained target detection model to the surveillance video source of the underground cavern dark dust scene to perform real-time visual recognition of multiple factors.

2. A multi-factor visual recognition method for underground cavern dark dust scenes as claimed in claim 1, characterized in that: The types of multi-element classification and labeling include construction workers, rock drilling trolleys, wet shotcrete trolleys, loaders, dump trucks, lining trolleys, steel bar trolleys, maintenance trolleys, invert trolleys, electric welders, vibrators, electric wires, ladders and scaffolding.

3. The multi-factor visual recognition method for underground cavern dark dust scenes according to claim 1, characterized in that: After step S1, the following steps are also included: The image data x 1 Import the self-calibration lighting module to perform self-calibration lighting processing. In the formula, t is the stage number, t≥1, x t is the input light intensity at stage t, G(·) represents self-calibration, v t The output light intensity after self-calibration in stage t, y is the self-calibration parameter, s t 、z t is the intermediate calculation amount, u t is the residual term of stage t, θ is the weight, K θ , H θ In order to introduce parameterized operators whose parameters can be learned, represents mapping, and F(·) represents lighting conversion; After T stages of self-calibration illumination processing, the output image data x T+1 ; Wherein, T is the total number of stages of the self-calibration lighting module.

4. A multi-factor visual recognition method for underground cavern dark dust scenes as claimed in claim 3, characterized in that: Calculate the total loss L during the self-calibration lighting process total , L total =αL f +βL s ; In the formula, α and β are positive balance parameters, L f For fidelity, L s To smooth the loss; Where N is the total number of pixels, i is the pixel ordinal number, A(i) represents the 5×5 pixels adjacent to the i-th pixel, and w i,j is the weight coefficient, is the imaging data of the i-th pixel; Where c is the image channel in the YUV color space, y i,c is the self-calibration parameter of the ith pixel in image channel c, is the intermediate calculation amount of the i-th pixel in the image channel c at stage t-1, and the standard deviation of the Gaussian kernel; By minimizing the total loss L total Determine the optimal weight coefficient w i,j .

5. The multi-factor visual recognition method for underground cavern dark dust scenes according to claim 1, characterized in that: The constructing of the target detection model further includes: Replace the concat module in Neck with the BiFPN module.

6. The multi-factor visual recognition method for underground cavern dark dust scenes according to claim 1, characterized in that: Step S4 includes: Step S41: input the image data set into the Backbone module; Step S411: In the Conv module, convolution, normalization and nonlinear processing are performed on each image data in sequence through the convolution layer, BN layer and activation function layer; Step S412: Input the nonlinearly processed image data F into the CBAMC3 module. Where M C (F) and M S (F) are the channel attention output feature map and the spatial attention output feature map, σ(·) represents the Sigmoid function, w1 and w0 are weight coefficients, They are the average pooling features of the channel attention mechanism and the spatial attention mechanism, respectively. They are the maximum pooling features of the channel attention mechanism and the spatial attention mechanism, respectively. 7×7 Represents a 7×7 convolution operation; Step S413: Output the spatial attention feature map M S (F) Input SPPF module; Concatenate all spatial attention output feature maps M S (F) is then input into the fully connected layer for dimensionality reduction to obtain the feature vector Step S42: Transform the feature vector Input Neck module for scale conversion, In the formula, is a feature vector of scale k×k, 3≤k≤7, Conv(·) represents convolution, and Resize(·) represents upsampling or downsampling; Step S43: inputting the scale-converted feature vector into the Head module; Set the target box through Anchors, and construct the corresponding detection box through K-means clustering method; Classify the detection box by combining the fully connected layer and the Softmax function through Classification; Use the fully connected layer through regression to regress the position and size of each detection box; Step S44: Set training parameters and start training until the preset number of rounds is completed.

7. A multi-factor visual recognition method for underground cavern dark dust scenes as claimed in claim 1 or 6, characterized in that: It is characterized in that After the training reaches the preset round, the training further includes: Calculate the maximum area enclosed by the PR curve and the coordinate axis, and take the corresponding weight as the optimal weight.

8. The multi-factor visual recognition method for underground cavern dark dust scenes according to claim 1, characterized in that: Setting the training parameters includes: The number of image data participating in the training in a single batch is set to 16, the momentum of the SGD optimizer is 0.937, the weight decay coefficient is 0.0005, the learning rate is 0.01, and the number of iterations is 100.

Images