Coal mine vehicle-mounted cargo type identification method and device based on improved yolov5

Abstract

The invention relates to a coal mine vehicle-mounted cargo type identification method and device based on improved yolov5. The method comprises the following steps: acquiring video data of each mine car at a loading point; preprocessing the obtained video data of each mine car at the loading point to obtain image data of each mine car at the loading point; constructing a training set, and performing artificial standard on the obtained image data to obtain the training set; constructing a network model, wherein the network model comprises an input feature separation unit, an intermediate representation change unit, a separation mechanism transformation unit and a convolution unit; model training: training a network model by using the constructed training set to obtain a trained network model; and the trained network model carries out type identification on goods on each mine car at the loading point.

Description

Technical Field

[0001] The present application relates to the field of intelligent coal mining industry and machine vision technology, and in particular to a method and device for identifying the type of cargo on a coal mine vehicle based on an improved YOLOv5. Background Art

[0002] AI identification of cargo on coal mine vehicles is to use artificial intelligence technology to identify and classify cargo on coal mine vehicles to achieve automated management and optimized transportation. First, AI identification can accurately monitor and identify cargo on coal mine vehicles to ensure that no contraband or dangerous goods are carried during transportation, which helps to ensure the safety of miners and equipment, reduce accidents, and enhance safety. Secondly, AI identification can quickly and accurately classify cargo on coal mine vehicles, avoiding the time-consuming and error-prone traditional manual inventory, and greatly improving work efficiency. At the same time, the transportation process can be optimized through real-time monitoring and data analysis to reduce empty load rate and transportation costs. Finally, by identifying and recording cargo on coal mine vehicles, it is possible to achieve fine management and traceability of resources, accurately calculate the workload of each miner and each vehicle, improve resource utilization efficiency, and analyze various data in the coal mine transportation process to provide a scientific basis for decision-making.

[0003] In 2016, Joseph et al. proposed YOLO. As the first formal single-stage target detector in the era of deep learning, this method completely abandoned the two-stage "region proposal + regression" detection mode, scaled the image to be tested to a uniform size and divided it into multiple grids, and then predicted the target category based on the grid where the target center is located, and output the detection result on the last convolutional layer. With the continuous optimization and adjustment of the network structure, YOLOV5 made a heavy debut, embedding the anchor box operation function in the input end of the network, and adaptively calculating the optimal anchor box value on each training set as needed. The Focus structure is added to the backbone network, and the slicing operation method is used to reduce the amount of calculation and parameters.

[0004] Since some coal mine vehicle cargo targets are small, dense and not obvious enough, such as clean coal, lump coal, coal slime, etc., YOLOV5 is not sensitive enough to small target detection and not strong enough to dense target detection, resulting in insufficient performance in detecting vehicle cargo types. Therefore, in view of the above problems, the present invention first collects seven types of coal mine vehicle cargo data through an AI camera, namely: clean coal, lump coal, gangue, mechanical cloth, cover cloth, coal slime and empty car. Before model training, the principal component analysis technology is introduced to amplify the image processing to extract the key features of the image more comprehensively. Secondly, YOLOV5 is selected as the pre-weight model for coal mine vehicle cargo type recognition, and the YOLOV5 backbone network structure is improved in this model: the input feature group is separated, and the SE attention mechanism is added to the convolution transformation before the separation mechanism, so that the network level is richer and more comprehensive. At the same time, the learning rate, number of iterations, batch processing size and optimization function are continuously iterated and optimized during the training process to compare and find the model with the optimal accuracy, and finally the Focal-loss loss function and the center point loss function are combined to jointly evaluate the model performance. Summary of the invention

[0005] Aiming at the problem that the coal mine vehicle cargo targets are small, dense and have unclear features, resulting in reduced model detection performance, the present invention provides a method and device for identifying the types of coal mine vehicle cargo based on improved YOLOv5, which deeply integrates computer vision technology with video surveillance rules to meet the real-time video recognition needs of coal mines.

[0006] The present application is implemented through the following technical solutions, a method for identifying the types of cargo on a coal mine vehicle based on improved yolov5, including:

[0007] Get video data of each minecart at the loading point;

[0008] Preprocessing the acquired video data of each mine car at the loading point to obtain image data of each mine car at the loading point;

[0009] Constructing a training set, and manually standardizing the acquired image data to obtain the training set;

[0010] Constructing a network model, the network model includes a separation input feature unit, an intermediate representation change unit, a separation mechanism transformation unit and a convolution unit;

[0011] Model training: Use the constructed training set to train the network model to obtain a trained network model;

[0012] The trained network model is used to identify the type of cargo on each mine car at the loading point.

[0013] Preferably, the loss function of the network model includes:

[0014] Where: L is the loss function of the network model, L b is the center loss function, L pt FocalLoss loss L = L b +δL pt

[0015] Loss function, δ is the balance scalar between the center loss function and the FocalLoss loss function;

[0016] in,

[0017]

[0018] x i is the i-category feature, is the i-th feature center, k represents the number of categories; y is the label value;

[0019]

[0020] Where: y is the label value. When the sample is a positive sample, the label value is 1, and when the sample is a negative sample, the label value is 0. To balance the weights of positive and negative samples, the value range is [0,1]; P t The probability of a certain category predicted by the representative model.

[0021] Preferably, the method for obtaining the image data of each mine car at the loading point comprises:

[0022] Obtain the frame image data of each mine car in the video data at the loading point, and perform vector conversion on each frame image to obtain a sequence matrix;

[0023] Decentralize the obtained sequence matrix to obtain a new sequence matrix;

[0024] Decompose the new sequence matrix to obtain the feature matrix;

[0025] Project the new sequence matrix through the feature matrix to obtain a new data set;

[0026] The new data set is decoded to obtain a new frame image, and an image data set of each mine car at the loading point is obtained.

[0027] Preferably, the network model is an improved yolov5 network model.

[0028] Preferably, the input feature separation unit divides the input image into K groups according to the number of channels to obtain K groups of images, and then splits each group of images to obtain multiple input feature groups.

[0029] Preferably, the intermediate representation change unit performs 1×1 convolution on each feature group and then performs attention processing, and finally performs 3×3 convolution to complete the intermediate feature change.

[0030] Preferably, the separation mechanism transformation unit fuses the feature groups processed by the intermediate representation unit to obtain a basis array of each K group, performs global average pooling on the basis array of each K group, performs attention weighting on the channels corresponding to each basis array, and finally completes the splicing to obtain the spliced ​​data. The spliced ​​data is convolved by the convolution unit to obtain the final feature image.

[0031] A coal mine vehicle cargo type identification device based on improved yolov5, comprising:

[0032] A video data acquisition module is used to acquire video data of each mine car at the loading point;

[0033] An image processing module is used to pre-process the acquired video data of each mine car at the loading point to obtain image data of each mine car at the loading point;

[0034] A network training module is used to manually standardize the acquired image data to obtain a training set;

[0035] Constructing a network model, the network model includes a separation input feature unit, an intermediate representation change unit, a separation mechanism transformation unit and a convolution unit;

[0036] Model training: Use the constructed training set to train the network model to obtain a trained network model;

[0037] The recognition module is used to use the trained network model to identify the type of cargo on each mine car at the loading point.

[0038] The beneficial effects of this application are:

[0039] Seven types of coal mine vehicle cargo data were collected through cameras. The principal component analysis technology was introduced before model training to enlarge and process the image to extract the key features of the image more comprehensively. YOLOV5 was selected as the pre-weighted model for coal mine vehicle cargo type recognition, and the YOLOV5 backbone network structure was improved in this model: the input feature group was separated, and the SE attention mechanism was added to the convolution transformation before the separation mechanism, making the network layer richer and more comprehensive. At the same time, the learning rate, number of iterations, batch processing size, and optimization function were continuously iterated and optimized during the training process to compare and find the model with the best accuracy. Finally, the Focal-loss loss function and the center point loss function were combined to jointly evaluate the model performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 This is a flow chart of a method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 of the present invention;

[0042] Figure 2 is a schematic diagram of the structure of a network model in an embodiment of the present invention;

[0043] Figure 3 is a schematic diagram of a mine truck carrying cargo in this embodiment;

[0044] Figure 4 It is a block diagram of a device for identifying the type of cargo on a coal mine vehicle based on improved yolov5 in an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the objectives, technical solutions and advantages of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0046] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0047] Example

[0048] like Figure 1 A method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 is shown, comprising:

[0049] Step S1, obtaining video data of each mine car at the loading point; when obtaining video data, it is necessary to obtain video data through an AI camera, including

[0050] S101, installing a camera. It is best to install the camera just above the loading point so that the collected data is suitable for later image annotation. If there are cameras in the actual coal mine application scenario and some cameras are not suitable for changing positions, the image can be collected normally. The later annotation method can be changed according to the image situation and adjusted to a method suitable for image data obtained at the camera installation angle;

[0051] S102, configure video storage, add camera configuration information to the local AI inference server through the camera manufacturer's private protocol, ONVI F open protocol or national standard GB28181 protocol, and enable NVR service for video storage, diversion forwarding, and diversion identification functions. Set the local hard disk as the storage path, and the specific parameters are: storage directory path and disk full coverage ratio: Set the full coverage ratio, and execute the full coverage strategy when the ratio is reached.

[0052] S103, configure the recording plan, set the recording time template to 24-hour monitoring, and then set the recording plan template, which mainly includes the bit rate, bit rate and recording retention period when the recording is saved. After the setting is completed, associate the time template with the corresponding recording monitoring point to realize the recording monitoring of the specific monitoring point at a specific time;

[0053] S104, set the task of intercepting the video stream, and set the video stream storage parameters in advance: video saving format, video stream capture time period, video stream frame number, etc. According to the specific time of coal mine cargo loading, capture the video stream of the vehicle cargo before and after the camera is loaded one minute to ensure data integrity and obtain the video data of the mine vehicle loading point.

[0054] S2, pre-processing the acquired video data of each mine car at the loading point to obtain image data of each mine car at the loading point; the pre-processing in this embodiment refers to data enhancement of the original data collected in step S1. Since some of the cargo on the vehicle is small and dense, this embodiment uses the principal component analysis method to decompose the video sequence, and then amplifies the amplitude of the decomposed low-frequency components. After a series of processing is completed, the video stream is framed to obtain image data; including:

[0055] S201. Convert the video sequence into a matrix;

[0056] Convert each frame of the video sequence into a vector and arrange these vectors into a matrix in chronological order: Assume that the video sequence contains N frames of images, and the vector of each image is represented by x i , where i is the index of the frame (i=1...N), the operation of converting the video sequence into a matrix can be expressed as:

[0057] X=[x1,x2,x3......x N ]

[0058] Thus, X is a matrix of size M x N, where M is the dimension of each image vector.

[0059] S202.Matrix centralization;

[0060] Subtract the average value of all elements in each column from the element value of each column to get Xcenter:

[0061] Xcenter=X-mean N

[0062] mean N =(1 / N)*sum(X,axis=0)

[0063] Among them, mean N is the mean of each column in the X matrix, axis=0 vertically, and N is the number of columns in the X matrix.

[0064] S203. Calculate the covariance matrix and decompose its eigenvalues, where the eigenvalues ​​represent the variance of the data set in the direction of the eigenvectors, and the eigenvectors represent the main direction of change of the data set in the corresponding direction;

[0065] C=(1 / N-1)*Xcenter*Xcenter T

[0066] C=V*A*V T

[0067] Where A is the diagonal matrix after eigenvalue decomposition, V is the eigenvector matrix, V T is the eigenvector transposed matrix, and Xcenter is the result of matrix centering in S202.

[0068] S204. Select the principal components, arrange the eigenvalues ​​in ascending order, and select the first K eigenvectors corresponding to the eigenvalues ​​to form a feature matrix V = [V1, V2···V K ].

[0069] S205. Low-frequency component amplification processing can achieve amplification effect by adjusting the size of the main component coefficient. A larger coefficient will lead to a more obvious amplification effect.

[0070] S206. Projection: Project the original matrix through the eigenvector to obtain a new data set after the dimensionality reduction of the original data set. Project the original matrix Xcenter through the eigenmatrix V to obtain the reduced-dimensional data set Q:

[0071] Q=V T *Xcenter

[0072] Among them, V is the eigenvector matrix, and Xcenter is the result of centering the S202 matrix.

[0073] After completing all the above steps, import the video stream into the OpenCV library for compilation to obtain frame images, and finally save them to the corresponding file directory of the server, filter the image data, and finally retain the image data of the cargo on the vehicle.

[0074] S207. Further enhance the amplified image data.

[0075] Call the specific functions in the Torchvision.transforms module to adjust the image's hue, saturation, rotation angle, scaling, shearing, perspective transformation, and other operations to make the enlarged image data more in line with the data training requirements. Specifically, first use the random cropping function to crop the input image, use the random rotation function to rotate it, then use the color transformation function to perform random color transformation, and then combine manual selection to exclude images of particularly poor quality, thereby improving the model's ability to recognize new data.

[0076] S3, constructing a training set, and manually standardizing the acquired image data to obtain a training set;

[0077] Use the labelimg tool to Figure 3 The image shown is rectangularly annotated to obtain the clean coal, lump coal, gangue and coal slime data. Due to the camera installation position angle problem in the actual scene, for the empty car, mechanical cloth and cover cloth data, if labelimg rectangular annotation is continued, redundant and irrelevant features will appear on the image data. Considering that the annotation method will have a certain impact on the key features extracted by the model in the later stage, in order to solve this problem, the present invention combines the use of labelme polygons to annotate the mechanical cloth, cover cloth and empty car data, and finally converts the annotation data format into a unified txt format adapted to the YOLOV5 model.

[0078] S4, constructing a network model, which includes a separation input feature unit, an intermediate representation change unit, a separation mechanism transformation unit and a convolution unit; in this embodiment, since some coal mine vehicle cargo targets are small, dense and have unclear features, such as clean coal, lump coal, coal slime, etc., YOLOV5 is not sensitive enough to small target detection and is not strong enough to dense target detection. Therefore, based on the YOLOV5 model structure, the present invention improves the YOLOV5 backbone network structure as follows. The schematic diagram of the improved main network structure is shown in the figure. Figure 2 As shown, the specific processing includes:

[0079] S401, separate input feature groups

[0080] like Figure 2 As shown in the figure, the input feature map is divided into K groups according to the channel dimension, each of which is represented as the kth group. These feature maps are used as the input base array. After the determination, a new parameter D is introduced to continue the splitting operation on the base array. D represents the number of divisions within the base array, and each split group is recorded as d. Therefore, according to the setting of the two parameters and the channel dimension, the total number of input feature groups becomes G = KD.

[0081] S402. Intermediate representation transformation;

[0082] Each individual feature group can perform a separate series of transformations {F1, F2···F G}, then the intermediate representation of each group is U i =F i (x)i∈{1,2···G}, in the intermediate representation of this embodiment, the SE module is added to each split basis array after the 1x1 convolution, and the channel dimension of the convolution feature map is adaptively processed. The hybrid attention mechanism module is used to improve the model's learning ability for key feature points; the channel attention module first performs an average pooling operation on the input feature map to obtain the average feature value of each channel. Then, the weight of each channel is learned through two fully connected layers, and finally a channel attention vector is obtained. By multiplying it with the input feature map, the feature maps of different channels are weighted. These weighted feature maps are then input into the later 3x3 convolution operation to better describe the features of the input image, thereby improving the performance of the model.

[0083] S403. Separation mechanism transformation split-attention

[0084] After the convolution transformation, the intermediate representation features are fused in a single basis array, and then fused by summing the elements across multiple segments to obtain the combined representation of each basis array. According to S401: the input feature map is divided according to the channel dimension to form a total of k input feature map basis arrays, and the kth basis array is represented as The specific expression is as follows:

[0085]

[0086] Among them, H, W, C are the size of the feature map, specifically the height, width and number of channels of the image, G = KD, K is the total number of input feature map base arrays, each base array is the kth group, D is the number of internal split groups of the input feature map base array, U m is the intermediate representation of each basis array.

[0087] After global average pooling, we get the cth component To reduce the dimension of the feature map output by the convolutional layer:

[0088]

[0089] Then, each channel is weighted by the channel soft attention mechanism, and the cth channel The calculation formula is as follows:

[0090]

[0091] in, It represents the attention weight of the corresponding channel after softmax normalization, and the weight range is (0,1). represents the feature component after global average pooling, k represents the number of basis arrays, D represents the number of internal splits of the basis array, and then the basis arrays are concatenated to obtain V:

[0092] V = concat {V 1 ,V 2 ···V K}

[0093] Then the current input and output are connected with the standard residual structure and used as the new feature output Q;

[0094] Q=V+θ(X)

[0095] Where θ(x) is the skip connection mapping in the residual block. After the subsequent convolution operation, the final feature map is obtained.

[0096] S5 model training: using the constructed training set to train the network model to obtain a trained network model;

[0097] When training the model, the original images in the constructed training set are used as input, and the annotated images are used as output for training. Before training, hyperparameters such as learning rate, batch size, and number of iterations need to be set in advance. During the training process, the main purpose is to introduce the activation function and loss function in the network training architecture;

[0098] In one embodiment, the network structure is YOLOV5, and the activation function used internally is the Mish function, which mainly introduces nonlinear elements into neurons to increase learning ability. Otherwise, each output is a linear function of the upper layer input.

[0099] S501, select optimizer and loss function

[0100] The optimization process is actually the process of minimizing the loss function. The smaller the loss function, the better the model performance. The model that uses only one loss function has poor performance. Therefore, the algorithm compares the effects of multiple loss functions, selects multiple appropriate loss functions, assigns weights to them, and jointly measures the quality of the model. The process of continuously adjusting parameters during model training is to minimize the loss function, so as to achieve the optimal model effect. In order to prevent the gradient from disappearing or exploding during model training, an L2 regularization term is added to the loss function.

[0101] S502, Model Evaluation

[0102] The basic parameters of accuracy are used as the evaluation standard. The loss function is mainly used in the training stage of the model. After each batch of training data is sent to the model, the predicted value is obtained through forward propagation. Then, the difference between the predicted value and the true value is calculated to optimize and adjust the model parameters to minimize the loss value, thereby achieving the purpose of learning.

[0103] Focal Loss is a loss function used to solve the problem of class imbalance. It adjusts the weights of positive and negative samples so that samples that are difficult to classify receive more attention during training. Center point loss is used to predict the center point position of the target box, helping the model to regress the position of the target box more accurately, thereby improving the accuracy of target detection. In this embodiment, the center loss function is combined with the focal-loss loss function to jointly supervise network training and jointly supervise model training to find the optimal parameters.

[0104] The following is the definition of the center loss function:

[0105]

[0106] Among them, L b is the center loss, x i is the category feature, is the feature center of the i-th category, and k represents the batch processing size. The Focal Loss function is defined as:

[0107]

[0108] Among them, L pt is the Focal Loss loss function, y is the label value, the positive sample label value is 1, and the negative sample label value is 0. To balance the weights of positive and negative samples, the value range is [0,1]; P t It represents the probability (i.e. confidence) of a certain category predicted by the model.

[0109] The loss function of the entire network is:

[0110] L=L b +δL pt

[0111] Among them, δ is the balance scalar of the two loss functions. The scalar value is continuously optimized during the retraining process to minimize L.

[0112] S6. Use the trained network model to identify the type of cargo on each mine car at the loading point. Specifically, the image captured by the camera is input into the network model, and the network model automatically identifies the cargo in the input image and outputs the identification result.

[0113] This embodiment is based on the coal mine vehicle cargo type recognition algorithm trained by the improved YOLOV5 model. Compared with the traditional algorithm, the algorithm has better robustness and accuracy by debugging the training parameters and network architecture. The improved model after training is more accurate than other training models. In the training process, the Focal-loss loss function and the center function are combined, and the results are more accurate than the only loss function.

[0114] It has improved the accuracy of identifying the types of cargo on coal mine vehicles, which has certain guiding significance for improving the coal mine management system. It can quickly and accurately classify and record the cargo on coal mine vehicles, avoiding the time-consuming and error-prone traditional manual inventory and greatly improving work efficiency. It can accurately monitor and identify the cargo on coal mine vehicles to ensure that no contraband or dangerous goods are carried during transportation. This helps to ensure the safety of miners and equipment, reduce accidents and enhance safety.

[0115] like Figure 4 As shown, this embodiment provides a coal mine vehicle cargo type identification device based on improved yolov5, including:

[0116] A video data acquisition module is used to acquire video data of each mine car at the loading point;

[0117] An image processing module is used to pre-process the acquired video data of each mine car at the loading point to obtain image data of each mine car at the loading point;

[0118] A network training module is used to manually standardize the acquired image data to obtain a training set;

[0119] Constructing a network model, the network model includes a separation input feature unit, an intermediate representation change unit, a separation mechanism transformation unit and a convolution unit;

[0120] Model training: Use the constructed training set to train the network model to obtain a trained network model;

[0121] The recognition module is used to use the trained network model to identify the type of cargo on each mine car at the loading point.

[0122] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the above-mentioned various optional implementations.

[0123] As another aspect, the present application also provides a computer-readable medium, which may be included in the electronic device described in the above embodiment; or may exist independently without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the above embodiment.

[0124] It should be noted that, although several modules or units of the equipment for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into being embodied by multiple modules or units.

[0125] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a touch terminal, or a network device, etc.) to execute the method according to the implementation methods of the present application.

[0126] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the embodiments disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application, which follow the general principles of the present application and include common knowledge or customary technical means in the art that are not disclosed in the present application.

[0127] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for identifying the type of cargo on a coal mine vehicle based on improved yolov5, characterized in that: include: Get video data of each minecart at the loading point; Preprocessing the acquired video data of each mine car at the loading point to obtain image data of each mine car at the loading point; Constructing a training set, and manually standardizing the acquired image data to obtain the training set; Constructing a network model, the network model includes a separation input feature unit, an intermediate representation change unit, a separation mechanism transformation unit and a convolution unit; Model training: Use the constructed training set to train the network model to obtain a trained network model; The trained network model is used to identify the type of cargo on each mine car at the loading point.

2. A method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 according to claim 1, characterized in that; The loss function of the network model includes: L=Lb+δLpt Where: L is the loss function of the network model, L b is the center loss function, L pt is the FocalLoss loss function, δ is the balance scalar between the center loss function and the FocalLoss loss function; in, x i is the i-category feature, is the i-th feature center, k represents the number of categories; y is the label value; Where: y is the label value. When the sample is a positive sample, the label value is 1, and when the sample is a negative sample, the label value is 0. To balance the weights of positive and negative samples, the value range is [0,1]; P t The probability of a certain category predicted by the representative model.

3. The method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 according to claim 1 is characterized in that: The method of obtaining the image data of each mine car at the loading point includes: Obtain the frame image data of each mine car in the video data at the loading point, and perform vector conversion on each frame image to obtain a sequence matrix; Decentralize the obtained sequence matrix to obtain a new sequence matrix; Decompose the new sequence matrix to obtain the feature matrix; Project the new sequence matrix through the feature matrix to obtain a new data set; The new data set is decoded to obtain a new frame image, and an image data set of each mine car at the loading point is obtained.

4. The method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 according to claim 1 is characterized in that: The network model is an improved yolov5 network model.

5. The method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 according to claim 1 is characterized in that: The input feature separation unit divides the input image into K groups according to the number of channels to obtain K groups of images, and then splits each group of images to obtain multiple input feature groups.

6. A method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 according to claim 5, characterized in that: The intermediate representation change unit performs 1×1 convolution on each feature group and then performs attention processing, and finally performs 3×3 convolution to complete the intermediate feature change.

7. The method for identifying the type of cargo on a coal mine vehicle based on improved yolov5 according to claim 5 is characterized by: The separation mechanism transformation unit fuses the feature groups processed by the intermediate representation unit to obtain a basis array of each K group, performs global average pooling on the basis array of each K group, performs attention weighting on the channels corresponding to each basis array, and finally completes the splicing to obtain the spliced ​​data. The spliced ​​data is convolved by the convolution unit to obtain the final feature image.

8. A coal mine vehicle cargo type identification device based on improved yolov5, characterized in that: include, A video data acquisition module is used to acquire video data of each mine car at the loading point; An image processing module is used to pre-process the acquired video data of each mine car at the loading point to obtain image data of each mine car at the loading point; A network training module is used to manually standardize the acquired image data to obtain a training set; Constructing a network model, the network model includes a separation input feature unit, an intermediate representation change unit, a separation mechanism transformation unit and a convolution unit; Model training: Use the constructed training set to train the network model to obtain a trained network model; The recognition module is used to use the trained network model to identify the type of cargo on each mine car at the loading point.

Images