Underground overhead man-riding device identification model training method and device and storage medium

Abstract

The embodiment of the invention discloses an underground monkey vehicle identification model training method and device and a storage medium, and the method comprises the steps: obtaining a training image set; inputting the training image set into a YOLOv8 model for training, convolution of a ninth-layer c2f module in the YOLOv8 model being deformable convolution, receiving convolution image features input by a convolution layer, and sequentially performing deformable convolution, separation and splicing operation on the convolution image features; determining the line width corresponding to each pixel point according to the printing proportionality coefficient; and carrying out iterative optimization on the YOLOv8 model, evaluating the optimized model, and taking the model meeting the evaluation standard as an underground monkey vehicle identification model. The method can improve the extraction of semantic information by a shallow network of the model, is suitable for various sitting posture details of miners, and improves the identification capability of the model for aerial cableway targets in different manned states.

Description

Technical Field

[0001] The present invention relates to the technical field of underground monkey car recognition, and in particular to an underground monkey car recognition model training method, device and storage medium. Background Art

[0002] Coal mine overhead passenger devices (commonly known as monkey cars) are one of the important equipment for underground transportation of people. The intelligent inspection technology of monkey cars is of great significance to improving the safety of coal mines, ensuring the efficiency of manned transportation, and reducing the number of people and improving efficiency. It is an important guarantee for safe and efficient transportation in coal mines. Due to the limitations of the complex underground environment and the level of early warning technology, monkey car accidents often occur frequently, seriously endangering the lives of miners and the safe and efficient production of coal mines. With the continuous acceleration of the intelligent construction of coal mines, the use of computer vision technology (Computer Vision) to observe the operation status of monkey cars in real time based on the monitoring screen underground can effectively avoid accidents.

[0003] In the process of realizing the present invention, the inventors discovered the following technical problems: since the underground images collected have poor lighting conditions, small targets, and are affected by different movements of the passengers, it is difficult to identify the monkey car using a neural network model. Summary of the invention

[0004] The embodiment of the present invention provides a method, device and storage medium for training a monkey car identification model in a mine, so as to solve the technical problem that it is difficult to identify monkey cars using a neural network model in the prior art.

[0005] In a first aspect, an embodiment of the present invention provides a method for training a monkey car recognition model in a mine, comprising:

[0006] Get the training atlas;

[0007] The training atlas is input into a YOLOv8 model for training, wherein the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, and the convolution image features input by the convolution layer are received, and the convolution image features are sequentially subjected to deformable convolution, separation, and splicing operations;

[0008] The ninth layer c2f convolution module includes:

[0009] A first variable convolution block and a second variable convolution block, wherein the first variable convolution block can perform a convolution operation according to the number of convolution kernels defined by an input, the second variable convolution block determines the number of convolution kernels of the second variable convolution block according to the number of convolution kernels of the first variable convolution block, and the residual module in the ninth layer c2f convolution module determines whether to enable or not according to the number of input channels and the number of output channels;

[0010] The YOLOv8 model is iteratively optimized, and the optimized model is evaluated, and the model that meets the evaluation criteria is used as the underground monkey car recognition model.

[0011] In a second aspect, an embodiment of the present invention further provides an underground monkey car recognition model training device, comprising:

[0012] An acquisition module is used to acquire a training atlas;

[0013] A training module, used for inputting the training atlas into a YOLOv8 model for training, wherein the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, receiving convolution image features input by the convolution layer, and sequentially performing deformable convolution, separation and splicing operations on the convolution image features;

[0014] The ninth layer c2f convolution module includes:

[0015] A first variable convolution block and a second variable convolution block, wherein the first variable convolution block can perform a convolution operation according to the number of convolution kernels defined by an input, the second variable convolution block determines the number of convolution kernels of the second variable convolution block according to the number of convolution kernels of the first variable convolution block, and the residual module in the ninth layer c2f convolution module determines whether to enable or not according to the number of input channels and the number of output channels;

[0016] The optimization module is used to iteratively optimize the YOLOv8 model and evaluate the optimized model, and use the model that meets the evaluation criteria as the underground monkey car recognition model.

[0017] In a third aspect, an embodiment of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute the underground monkey car recognition model training method provided in the above embodiment.

[0018] The underground monkey car recognition model training method, device and storage medium provided by the embodiment of the present invention are as follows: a training atlas is obtained; the training atlas is input into the YOLOv8 model for training; the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, the convolution image features input by the convolution layer are received, and the convolution image features are sequentially subjected to deformable convolution, separation and splicing operations; the ninth layer c2f convolution module includes: a first variable convolution block and a second variable convolution block, the first variable convolution block can perform a convolution operation according to the number of convolution kernels defined by the input, the second variable convolution block determines the number of convolution kernels of the second variable convolution block according to the number of convolution kernels of the first variable convolution block, and the residual module in the ninth layer c2f convolution module determines whether to enable according to the number of input channels and the number of output channels; the YOLOv8 model is iteratively optimized, and the optimized model is evaluated, and the model that meets the evaluation criteria is used as the underground monkey car recognition model. By optimizing the YOLOv8 model structure and using the first and second variable convolution blocks to address the poor clarity and small image proportion of the monkey car image, the model's shallow network can be improved in extracting semantic information, adapting to the various sitting posture details of miners, and increasing the model's ability to recognize monkey car targets in different manned states. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0020] Figure 1 It is a flow chart of the underground monkey car recognition model training method provided by the first embodiment of the present invention;

[0021] Figure 2 It is a structural schematic diagram of the underground monkey car recognition model training device provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0023] Embodiment 1

[0024] Figure 1 : is a flow chart of the underground monkey car recognition model training method provided by the first embodiment of the present invention. This embodiment is applicable to the case of training the YOLOv8 model for identifying underground monkey cars. The method can be executed by the underground monkey car recognition model training method device, and specifically includes the following steps:

[0025] Step 110, obtain a training atlas.

[0026] The training atlas can be acquired by taking pictures with a variety of intrinsically safe explosion-proof cameras for mining, and the image acquisition locations may include two monkey car heads, one running middle, and two monkey car tails. According to the collected monkey car image data set, the monkey car images are annotated into two types of labels, namely, the monkey car state with no one riding (HC_nominer) and the monkey car state with someone riding (HC_miner).

[0027] Since underground explosion-proof cameras are used as the main way to obtain image data sets, most of the collected image data have disadvantages such as low light, blur, and severe occlusion. Therefore, it is necessary to process the collected images. Considering the influence of coal dust and low light in the underground environment, adaptive histogram equalization can be used to enhance the contrast of underground images and weaken the negative environmental impact of underground environments; adaptive histogram equalization is to divide the input image into blocks of preset sizes, and then equalize each pixel through the histogram of pixels in a rectangular range around it, so as to equalize the image grayscale and enhance the edge of the image.

[0028] For example, the number of pixels in the input image can be recorded as x, and the value is [0, L-1], where L is the discrete grayscale level in the dynamic range. r (r k ) can be expressed as:

[0029]

[0030] The grayscale cumulative distribution function Sk can be expressed as:

[0031]

[0032] The histogram equalization transformation function can be expressed as: r′ k =round((L-1),S K ),K=0,1,2,…,L-1, where: r k is the kth gray level; n k is the number of pixels appearing in the image; r′ k is the gray level after histogram equalization; round(·) is the rounding operation;

[0033] Taking into account the occlusion and overlap of monkey cars in actual scenarios, this embodiment uses random rectangular frame occlusion to simulate the situation where the recognition target is blocked or partially blocked by other objects to enhance the recognition ability of the monkey car underground, so that the model can better adapt to various complex scenes and handle occlusion. Optionally, the random rectangular occlusion process includes randomly selecting the position of the rectangle, randomly determining the size of the rectangle, and randomly generating a rectangular occlusion. First, a random function is used to randomly select a position from the image as the upper left corner coordinate of the rectangle; then, a specific range that does not exceed the size of the detection target is set to control the random width and height of the rectangle; finally, according to the selected position and size, the corresponding area in the image is covered with a rectangular occlusion. Black, white or other colors can be used to fill the rectangular area; considering the small amount of data, this embodiment uses vertical flipping and horizontal flipping to expand the data set while reducing the impact of noise in the image on monkey car recognition; the two flipping operations can be implemented through the index of pixels. For vertical flipping, the upper half of the row pixels of the image can be traversed by looping and swapped with the lower half of the row pixels; for horizontal flipping, the column pixels of the image can be traversed by looping and swapped with the corresponding inverse column pixels; four types of enhancement operations can also be selected: adaptive histogram equalization (Clahe), random rectangular box occlusion (Cutout), vertical flip (V-Flip), and horizontal flip (H-Flip). The training atlas is generated through the above operations.

[0034] Step 120, input the training atlas into the YOLOv8 model for training, the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, receives the convolution image features input by the convolution layer, and performs deformable convolution, separation and splicing operations on the convolution image features in sequence.

[0035] The main idea of ​​the YOLO algorithm is to divide the feature map into S×S grids, each grid is responsible for detecting the target that falls into it, and predict the bounding box, location confidence and all category probabilities of the targets contained in all grids at one time. Its specific structure is as follows:

[0036] The information of each layer of the YOLOv8 backbone network is as follows: Layer 0 CBL (conv-bn-silu) channel: 64 size: 320 * 320 Layer 1 CBL (conv-bn-silu) channel: 128 size: 160 * 160; Layer 2 C2f * 3 channel: 128 size: 160 * 160 shortcut = True; Layer 3 CBL (conv-bn-silu) channel: 256 size: 80 * 80; Layer 4 C2f * 6 channel: 256 size: 80 * 80 shortcut = True; Layer 5 CBL (conv-bn-si lu)channel:512size:40*40; the 6th layer C2f*6channel:512size:40*40shortcut=True; the 7th layer CBL(conv-bn-silu)channel:1024size:20*20; the 8th layer C2f*3channel:1024size:20*20shortcut=True; the 9th layer SPPF(k=5)channel:1024size:20*20; the 3rd to 8th layers are consistent with the 1st and 2nd layers, repeating 6-6-3 Bottleneck modules respectively. The 9th layer is a pooling layer, using the SPPF structure, multi-scale fusion features

[0037] The information of each layer of YOLOv8 neck is as follows:

[0038] Layer 10 upsample channel:1024 size:40*40; Upsampling starts at the 10th layer, using the nearest neighbor difference method to enlarge the feature map. Layer 11 concat channel:1536 size:40*40 Layer 6 output + layer 10 output = 512*40*40 + 1024*40*40 = 1536*40*40; Layer 12 C2f*3 channel:512 size:40*40 shortcut = False The 12th layer is similar to the C2F in the backbone, except that shortcut = False, the residual structure is cancelled, and the output size is from 1536*40*40 to 512*40*40 Layer 13 upsample channel:512 size:80*80; Layer 14 concat channel:768 size:80*80 The 14th layer is the output of the 4th layer + the output of the 13th layer

[0039] =256*80*80+512*80*80=768*80*80; Layer 15 C2f*3 channel:256 size:80*80 shortcut=False; Layers 13-15 are the same as layers 10-12; Layer 16 CBL (conv-bn-silu) channel:256 size:40*40; Layer 17 concat channel:768 size:40*40

[0040] The 17th layer is the output of the 12th layer + the output of the 16th layer = 512*40*40 + 256*40*40 = 768*40*40; the 18th layer C2f*3channel:512size:40*40shortcut=False; the 19th layer CBL(conv-bn-silu)channel:512size:20*20; the 20th layer concat channel:1536size:20*20; the 20th layer is the output of the 9th layer + the output of the 19th layer; the 21st layer C2f*3channel:1024size:20*20shortcut=False; the 16th-21st layers are the same as the 10th-12th layers except that they are changed from upsampling to downsampling. The YOLOv8 detection header information is as follows: the 22nd layer Detect channel:64+ncsize:80*80 40*40 20*20.

[0041] When using the YOLOv8 model for training, since the size of the monkey car in the images in the training atlas is variable, special processing is required to extract the semantic information in the training images, strengthen the model's learning of the coupling characteristics of the monkey car and miners, and adapt to the various sitting details of the miners.

[0042] The initialization parameters of the ninth layer of deformable convolution can be pre-set. The initialization parameters include (c1, c2, shortcut, g, k, e), which define the number of input channels c1, the number of output channels c2, whether to use shortcut connection, the number of groups g, the convolution kernel size k and the expansion factor e of the module.

[0043] Create a DCNv2 or Conv object based on the input convolution kernel size k[0] and k[1], respectively. If k[0] is equal to 3, use the DCNv2 object for convolution; otherwise, use the Conv object for convolution. Similarly, if k[1] is equal to 3, use the DCNv2 object for convolution; otherwise, use the Conv object for convolution. The parameters of the DCNv2 object are the number of input channels c1, the number of output channels c_, and the convolution kernel size k[0] or k[1], and the parameters of the Conv object are the number of input channels c_, the number of output channels c2, and the convolution kernel size k[0] or k[1]. In this way, different convolution operations can be flexibly selected according to specific task requirements, thereby improving the performance and accuracy of the model in the monkey car detection task.

[0044] Using the init function, depending on whether the input convolution kernel size k is 3, choose to use the DCNv2 operation or the ordinary convolution operation, and define the cv1 and cv2 layers respectively. The function of the cv1 and cv2 convolution layers is to perform feature extraction and channel number transformation, thereby improving the feature representation ability and performance of the model; in the cv1 layer, a 1×1 convolution is used to convert the input channel number c1 to twice the number of hidden channels self.c; the channel number transformation through the 1×1 convolution operation can increase the expressive power and diversity of the features. Correspondingly, in the cv2 layer, the (2+n) times the number of hidden channels self.c is converted to the output channel number c2; the hidden channel number is converted to the output channel number c2 through the convolution operation to obtain the feature representation;

[0045] Using the forward function, depending on whether shortcut and c1 are equal to c2, choose whether to use shortcut connection, then forward propagate through cv1 and cv2 layers, and add the result to the input x to get the output.

[0046] By introducing adaptive convolution kernel discrimination and selecting different operations for different input convolution kernel sizes, the structure of the model can be flexibly adjusted according to specific task requirements, thereby improving the performance and accuracy of the model in the monkey car detection task.

[0047] Optionally, the first variable convolution block and the second variable convolution block in the YOLOv8 model can also be optimized, and the first variable convolution block and the second variable convolution block are provided with: an offset layer, the offset layer convolution generates 2N channels through input features, corresponding to N two-dimensional offsets, where the two dimensions correspond to the X direction and the Y direction respectively, to obtain a learnable offset; the remaining N channels are sent to the sigmoid layer to obtain a modulation scalar, the number of channels N corresponds to the weight of the convolution kernel receptive field, and the modulation scalar is used to control the weight of the convolution kernel at different positions, thereby changing the size and shape of the receptive field of the convolution operation.

[0048] Deformable convolution introduces learnable offsets and modulation scalars based on traditional convolution, which can effectively solve the problem that the fixed grid in traditional convolution cannot capture the complex spatial changes of the detected object. The DCNv2 module is mainly composed of two independent convolution blocks, with a total output channel of 3N. The convolution of the offset layer generates 2N channels through input features, corresponding to N two-dimensional offsets (one in the x-direction and one in the y-direction), and obtains the learnable offset; the learnable offset is used to dynamically adjust the position of the receptive field to better capture the details and contextual information of the target object; the remaining N channels are sent to the sigmoid layer to obtain the modulation scalar, and the number of channels N corresponds to the weight of the convolution kernel receptive field;

[0049] By using the sigmoid function to map these scalars to a range between 0 and 1, the weights of the convolution kernel at different positions can be controlled, thereby changing the size and shape of the receptive field of the convolution operation; the learnable offset and modulation scalar are set to initial values ​​0 and 0.5, respectively, and updated using bilinear interpolation back propagation; through back propagation, the model can automatically learn the values ​​of the offset and modulation scalar suitable for the current task based on the gradient information of the loss function.

[0050] The deformable convolution formula is as follows:

[0051] Where: y(p) is the output; w k is the weight of the kth position of the convolution kernel; k is the number representing the position in the convolution grid; N is the number of feature map channels; x is the input feature; p is each position in the output feature map; p k is the element at the kth position; Δp k is the offset of the kth receptive field position; Δm k is the modulation scalar at the kth position.

[0052] Exemplarily, the YOLOv8 model also includes: a coordinate attention mechanism module, which includes: a dimensionality reduction unit, which is used to reduce the dimensionality of the height and width information of the input feature map to obtain a reduced dimensionality feature; a height convolution layer and a width convolution layer, which are used to obtain a height attention coefficient and a width attention coefficient respectively; a splicing processing unit, which is used to splice the height pooling result and the width pooling result using the height attention coefficient and the width attention coefficient to obtain an attention map, and process the attention map y; a segmentation unit, which is used to segment the processed attention map to obtain a height attention map and a width attention map, and transpose the width attention map in different dimensions respectively, and convolve the height attention map and the width attention through a convolution layer, and activate the convolution processing result using an activation function.

[0053] First, the height and width of the input are averagely pooled by the adaptive average pooling layer nn.AdaptiveAvgPool2d to obtain the height pooling result x_h and the width pooling result x_w. The purpose is to reduce the dimension of the height and width information of the input feature map to obtain a more compact feature representation for subsequent processing operations. Then, a 1×1 convolution layer self.conv1 is used to reduce the number of input channels inp to mip, reducing the number of parameters and computational complexity of the model, and at the same time, more abstract and high-level feature representations can be extracted. The 1×1 convolution layer is equivalent to a linear transformation of each pixel of the input. Here, the number of input channels is reduced to mip by a 1×1 convolution layer, that is, the dimension of each channel of the input feature map is adjusted. The calculation method of mip is max(8,inp / / reduction), where reduction is a hyperparameter used to control the degree of dimensionality reduction. If inp / / reduction is less than 8, the value of mip is 8, otherwise the value of mip is inp / / reduction.

[0054] The reduced-dimensional features can be processed by the batch normalization layer self.bn1 and the activation function h_swish(); the mean of each feature dimension is made to be 0 and the variance is 1, thereby accelerating the training of the model and improving the robustness of the model; finally, the height attention coefficient a_h and the width attention coefficient a_w are obtained respectively through two 1×1 convolutional layers self.conv_h and self.conv_w; the forward function is used as the forward propagation function of the coordinate attention module to implement the forward calculation process of the model; the input x is saved as the identity connection to obtain the dimension information of the input x, including the number of batches n, the number of channels c, the height h and the width w; the height pooling result x_h and the width pooling result x_w are concatenated on dimension 2 to obtain the attention map y, and the attention map y is processed by the convolution layer self.conv1, the batch normalization layer self.bn1 and the activation function self.act; the results of feature extraction and nonlinear transformation of the attention map y can also be obtained to further enrich and enhance the model's ability to extract important areas of the monkey car;

[0055] The processed attention map y is split on dimension 2 to obtain the height attention map x_h and the width attention map x_w. The width attention map x_w is transposed on dimensions 2 and 3. The height attention map x_h and the width attention map x_w are processed by the convolutional layers self.conv_h and self.conv_w, and the results are sigmoid activated. The role of the splitting process includes: separating the attention of height and width: through the splitting process, the original attention map y is decomposed into the height attention map x_h and the width attention map x_w, so that the model can pay attention to the information of height and width respectively. In this way, the weight distribution of the feature map in height and width can be more finely controlled, thereby improving the model's perception of the monkey car target features at different time scales. After the splitting process, the width attention map x_w is transposed on dimensions 2 and 3. The purpose of this is to make the width attention map have the same dimensions in height and width, which is convenient for subsequent convolution operations. The input x is multiplied by the width attention coefficient a_w and the height attention coefficient a_h to get the output. This layer of attention module is added before the three different scale feature detection anchors; multi-scale dependencies between networks are established to increase the model's extraction of small target monkey car features, filter background interference, and enable the model to adaptively adjust the detection anchor weights according to the context information of feature maps of different scales, so as to achieve the advancement of deep network semantic features and position information; coordinate attention mechanism is a lightweight and efficient attention mechanism, which mainly embeds position information into channel attention, so that the network can obtain information from a larger area and avoid introducing large overhead. First, the input image is globally averaged in both height and width directions. The formula is as follows:

[0056]

[0057] Where: is the cth channel output with height h; is the output of the cth channel with a width of w; W and H are the width and height of the feature map respectively; xc is the input value of the cth channel. The feature maps in the height and width directions after pooling are concatenated, normalized, convolved, and nonlinearly activated, and then separated into feature maps in the width and height directions through separation operations. After convolution, average pooling, and sigmoid activation functions, the attention weights gw and gh in the width and height directions are obtained. Finally, the feature map with attention weights in the width and height directions is obtained through multiplication weighted calculation on the original feature map. The formula is as follows:

[0058] Where: yc is the output feature map of the CA module; is the attention weight of the cth channel when the height is h; is the attention weight of the cth channel when the width is w.

[0059] Step 130, iteratively optimize the YOLOv8 model, evaluate the optimized model, and use the model that meets the evaluation criteria as the underground monkey car recognition model.

[0060] In this embodiment, the optimizer can use stochastic gradient descent with momentum combined with WarmUp (Adamw), with a learning rate of 0.001, a momentum of 0.937, and a weight decay coefficient of 0.0005 to control the regularization strength.

[0061] The model performance evaluation includes precision (Precision, P), recall (Recall, R), average precision (Average precision, AP), mean average precision (mAP), and detection frame rate per second (FPS)

[0062] P=TP / TP+FP

[0063] R=TP / TP+FN

[0064]

[0065] FPS=1 / t

[0066] Where: P and R are precision and recall respectively; TP, FN and FP are the number of positive check boxes, missed check boxes and false check boxes respectively; N is the number of detection categories; t is the time for detecting each image, unit is s.

[0067] The loss function can use a combination of classification and bounding box regression decoupled loss, which can effectively improve the model weight utilization rate, so that the label can better guide the model's ability to distinguish and learn different categories. Among them, the classification loss uses binary cross entropy (BCE), each category is judged as "whether it is this category", and the output is the confidence level. The formula is as follows:

[0068]

[0069] Where: N is the total number of samples; yi is the model prediction value at the i-th pixel position; pi is the actual value marked at the i-th pixel position.

[0070] The bounding box regression loss uses CIOU+DFL coupled loss. Compared with IOU, CIOU considers the overlapping area, center point distance, and aspect ratio, solving the problems of loss equal to 0, slow convergence, and low accuracy when the bounding boxes do not intersect. The penalty term and trade-off vector are shown in the following formula:

[0071]

[0072] The aspect ratio parameter and CIOU loss expression are shown in the following formula:

[0073]

[0074] L ClOU =1-IO U+R ClOU

[0075] Where: 2 (b,b gt ) are the Euclidean distances between the center points of the prediction box and the annotation box respectively; c is the minimum diagonal distance between the circumscribed rectangles of the prediction box and the annotation box; αv is the influence factor, which represents the fitting value of the aspect ratio of the prediction box and the aspect ratio of the annotation box; w gt and h gt are the width and height of the annotation box respectively; w and h are the width and height of the prediction box respectively.

[0076] DFL optimizes the probability of the two positions closest to the label, one on the left and one on the right, in the form of cross entropy, so that the network can focus on the distribution of the target position and the adjacent area more quickly. The formula is as follows:

[0077]

[0078] Where: y is the actual value; y i and i+1 S is the predicted value of the two positions closest to the label, one on the left and one on the right; i and S i+1 y i and i+1 The corresponding prediction confidence.

[0079] This embodiment obtains a training atlas; inputs the training atlas into a YOLOv8 model for training, wherein the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, receives the convolution image features input by the convolution layer, and sequentially performs deformable convolution, separation and splicing operations on the convolution image features; the ninth layer c2f convolution module includes: a first variable convolution block and a second variable convolution block, wherein the first variable convolution block can perform a convolution operation according to the number of convolution kernels defined by the input, the second variable convolution block determines the number of convolution kernels of the second variable convolution block according to the number of convolution kernels of the first variable convolution block, and the residual module in the ninth layer c2f convolution module determines whether to enable or not according to the number of input channels and the number of output channels; the YOLOv8 model is iteratively optimized, and the optimized model is evaluated, and the model that meets the evaluation criteria is used as the underground monkey car recognition model. By optimizing the YOLOv8 model structure and using the first and second variable convolution blocks to address the poor clarity and small image proportion of the monkey car image, the model's shallow network can be improved in extracting semantic information, adapting to the various sitting posture details of miners, and increasing the model's ability to recognize monkey car targets in different manned states.

[0080] In a preferred implementation of this embodiment, the method may further include the following step: using the trained underground monkey car recognition model to recognize the monkey car in the collected image. Using the trained underground monkey car recognition model, the underground monkey car can be accurately recognized for complex image conditions.

[0081] Embodiment 2

[0082] Figure 2 : is a schematic diagram of the structure of the underground monkey car recognition model training device provided by the second embodiment of the present invention, such as Figure 2 As shown, the device comprises:

[0083] An acquisition module 210 is used to acquire a training atlas;

[0084] A training module 220 is used to input the training atlas into a YOLOv8 model for training, wherein the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, receives the convolution image features input by the convolution layer, and sequentially performs deformable convolution, separation and splicing operations on the convolution image features;

[0085] The ninth layer c2f convolution module includes:

[0086] A first variable convolution block and a second variable convolution block, wherein the first variable convolution block can perform a convolution operation according to the number of convolution kernels defined by an input, the second variable convolution block determines the number of convolution kernels of the second variable convolution block according to the number of convolution kernels of the first variable convolution block, and the residual module in the ninth layer c2f convolution module determines whether to enable or not according to the number of input channels and the number of output channels;

[0087] The optimization module 230 is used to iteratively optimize the YOLOv8 model and evaluate the optimized model, and use the model that meets the evaluation criteria as the underground monkey car recognition model.

[0088] The underground monkey car recognition model training device provided in this embodiment performs wavelet transform denoising on the printed image, and reconstructs the denoised signal in a standardized manner according to the grayscale value of the image pixel according to the grayscale value assignment interval standard, so that the pixel points in the standardized reconstructed image are set to the standard reference value corresponding to the value assignment interval range according to the grayscale value corresponding to the value assignment interval range; the line width corresponding to each pixel point is determined according to the printing scale coefficient; the concave and convex line type of the pixel is determined according to the standard reference value of each pixel after reconstruction, and the concave and convex line type is the line type formed by the intersection of concave points and convex points; the thickness of the printed image is expanded according to the selected image thickness to generate a printed image; the path planning of the strokes of the printed image is generated, and then the coordinates of each turning point are determined, and a 3D printing code is generated according to the coordinates of each turning point. By denoising and transforming the image, the high-frequency feature part in the image is highlighted, and different concave and convex line types are generated for the high-frequency feature part according to different mapping relationships, and the concave and convex line types are used to reflect the three-dimensional changes of the image features. Without complex modeling and editing, printing and coding can be generated and 3D printing can be performed directly.

[0089] Based on the above embodiments, the first variable convolution block and the second variable convolution block include:

[0090] An offset layer, wherein the offset layer convolution generates 2N channels through input features, corresponding to N two-dimensional offsets, where the two dimensions correspond to the X direction and the Y direction respectively, to obtain a learnable offset;

[0091] The remaining N channels are sent to the sigmoid layer to obtain the modulation scalar. The number of channels N corresponds to the weight of the convolution kernel receptive field. The modulation scalar is used to control the weight of the convolution kernel at different positions, thereby changing the size and shape of the receptive field of the convolution operation.

[0092] On the basis of the above embodiments, the size and shape of the receptive field of the convolution operation are changed by the following method:

[0093] Among them, y(p) is the output; w kis the weight of the kth position of the convolution kernel; k is the number representing the position in the convolution grid; N is the number of feature map channels; x is the input feature; p is each position in the output feature map; p k is the element at the kth position; Δp k is the offset of the kth receptive field position; Δm k is the modulation scalar at the kth position.

[0094] Based on the above embodiments, the device further includes:

[0095] The YOLOv8 model also includes:

[0096] A coordinate attention mechanism module, wherein the coordinate attention mechanism module comprises:

[0097] A dimension reduction unit is used to reduce the dimension of the height and width information of the input feature map to obtain a reduced dimension feature;

[0098] The height convolution layer and the width convolution layer are used to obtain the height attention coefficient and the width attention coefficient respectively;

[0099] A splicing processing unit, used to splice the height pooling result and the width pooling result using the height attention coefficient and the width attention coefficient to obtain an attention map, and process the attention map y;

[0100] The segmentation unit is used to segment the processed attention map to obtain a high attention map and a wide attention map, and transpose the wide attention map in different dimensions respectively, and convolve the high attention map and the wide attention map through a convolution layer, and activate the convolution result using an activation function.

[0101] Based on the above embodiments, the acquisition module includes:

[0102] The acquisition unit is used to collect raw image data using an underground explosion-proof camera;

[0103] The enhancement unit is used to perform adaptive histogram equalization on the original image data to achieve contrast enhancement and obtain an enhanced image.

[0104] Based on the above embodiments, the acquisition module further includes:

[0105] The occlusion unit is used to perform random rectangular occlusion on the enhanced image and use the image after random rectangular occlusion as a training atlas.

[0106] Based on the above embodiments, the device further includes:

[0107] The recognition module is used to recognize the monkey car in the collected image using the trained underground monkey car recognition model.

[0108] The underground monkey car recognition model training device provided in the embodiment of the present invention can execute the underground monkey car recognition model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0109] Embodiment 3

[0110] Embodiment 3 of the present invention further provides a storage medium comprising computer executable instructions, which, when executed by a computer processor, are used to execute any of the underground monkey car recognition model training methods provided in the above embodiments.

[0111] The computer storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0112] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, which carry computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0113] Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0114] Computer program code for performing the operations of the present invention may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or device. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0115] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present invention, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A method for training a monkey car recognition model in a mine, characterized in that: include: Get the training atlas; The training atlas is input into a YOLOv8 model for training, wherein the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, and the convolution image features input by the convolution layer are received, and the convolution image features are sequentially subjected to deformable convolution, separation, and splicing operations; The ninth layer c2f convolution module includes: A first variable convolution block and a second variable convolution block, wherein the first variable convolution block can perform a convolution operation according to the number of convolution kernels defined by an input, the second variable convolution block determines the number of convolution kernels of the second variable convolution block according to the number of convolution kernels of the first variable convolution block, and the residual module in the ninth layer c2f convolution module determines whether to enable or not according to the number of input channels and the number of output channels; The YOLOv8 model is iteratively optimized, and the optimized model is evaluated, and the model that meets the evaluation criteria is used as the underground monkey car recognition model.

2. The method according to claim 1, characterized in that The first variable convolution block and the second variable convolution block include: An offset layer, wherein the offset layer convolution generates 2N channels through input features, corresponding to N two-dimensional offsets, where the two dimensions correspond to the X direction and the Y direction respectively, to obtain a learnable offset; The remaining N channels are sent to the sigmoid layer to obtain the modulation scalar. The number of channels N corresponds to the weight of the convolution kernel receptive field. The modulation scalar is used to control the weight of the convolution kernel at different positions, thereby changing the size and shape of the receptive field of the convolution operation.

3. The method according to claim 2, characterized in that The change in the receptive field size and shape of the convolution operation is achieved using the following method: Among them, y(p) is the output; w k is the weight of the kth position of the convolution kernel; k is the number representing the position in the convolution grid; N is the number of feature map channels; x is the input feature; p is each position in the output feature map; p k is the element at the kth position; Δp k is the offset of the kth receptive field position; Δm k is the modulation scalar at the kth position.

4. The method according to claim 1, characterized in that: The YOLOv8 model also includes: A coordinate attention mechanism module, wherein the coordinate attention mechanism module comprises: A dimension reduction unit is used to reduce the dimension of the height and width information of the input feature map to obtain a reduced dimension feature; The height convolution layer and the width convolution layer are used to obtain the height attention coefficient and the width attention coefficient respectively; A splicing processing unit, used to splice the height pooling result and the width pooling result using the height attention coefficient and the width attention coefficient to obtain an attention map, and process the attention map y; The segmentation unit is used to segment the processed attention map to obtain a high attention map and a wide attention map, and transpose the wide attention map in different dimensions respectively, and convolve the high attention map and the wide attention map through a convolution layer, and activate the convolution result using an activation function.

5. The method according to claim 1, characterized in that The obtaining of the training atlas comprises: Use underground explosion-proof cameras to collect raw image data; Adaptive histogram equalization is performed on the original image data to achieve contrast enhancement, thereby obtaining an enhanced image.

6. The method according to claim 5, characterized in that The obtaining of the training atlas further includes: The enhanced image is randomly occluded with a rectangle, and the image after random rectangular occlusion is used as a training atlas.

7. The method according to claim 1, characterized in that The method further comprises: The trained underground monkey car recognition model is used to identify the monkey car in the collected images.

8. An underground monkey car recognition model training device, characterized in that: include: An acquisition module is used to acquire a training atlas; A training module, used for inputting the training atlas into a YOLOv8 model for training, wherein the ninth layer c2f module convolution in the YOLOv8 model is a deformable convolution, receiving convolution image features input by the convolution layer, and sequentially performing deformable convolution, separation and splicing operations on the convolution image features; The ninth layer c2f convolution module includes: A first variable convolution block and a second variable convolution block, wherein the first variable convolution block can perform a convolution operation according to the number of convolution kernels defined by an input, the second variable convolution block determines the number of convolution kernels of the second variable convolution block according to the number of convolution kernels of the first variable convolution block, and the residual module in the ninth layer c2f convolution module determines whether to enable or not according to the number of input channels and the number of output channels; The optimization module is used to iteratively optimize the YOLOv8 model and evaluate the optimized model, and use the model that meets the evaluation criteria as the underground monkey car recognition model.

9. A storage medium containing computer executable instructions, characterized in that: The computer executable instructions, when executed by a computer processor, are used to execute the underground monkey car recognition model training method as described in any one of claims 1-6.

Images