A speed regulation method and system based on video deep learning algorithm

Abstract

The present application provides a speed regulation method based on a video deep learning algorithm. The speed regulation method based on a video deep learning algorithm includes: obtaining image information of a conveyor belt and the amount of coal; obtaining the grading information of the amount of coal on the current conveyor belt according to the image information; judging whether the grading information of the amount of coal on the current conveyor belt matches the conveying speed of the current conveyor belt, and if not, generating a speed regulation signal according to the grading information of the amount of coal on the current conveyor belt and transmitting it to the controller of the conveyor belt, and the controller changes the conveying speed of the conveyor belt according to the speed regulation signal. The present application grades the amount of coal on the conveyor belt by image recognition to obtain the grading information of the amount of coal, and controls the speed of the conveyor belt by the grading information of the amount of coal, so that the speed of the conveyor belt can always match the amount of coal, thereby saving improper energy consumption.

Description

Technical Field

[0001] The present application belongs to the field of coal transportation technology, and in particular relates to a speed regulation method based on a video deep learning algorithm, a coal transportation belt speed regulation device based on a video deep learning algorithm, and a coal transportation system. Background Art

[0002] As one of the most important basic energy sources in my country, coal occupies an indispensable strategic position in the national economy. my country's coal industry occupies a leading position in the world, with an annual coal production of more than 200 billion tons. However, the transportation link after coal production is a big problem. The distance from underground coal mining to coal bunker is long and energy consumption is high. The energy consumption of conveyor belt conveyor to transport raw coal is very huge, and the energy consumed accounts for about 30% of the total energy consumption of coal mines.

[0003] The prior art usually performs real-time frequency conversion speed regulation on the belt conveyor according to the amount of coal on the belt. For example, it is usually judged by manual observation or the weight on the belt.

[0004] Therefore, it is desired to have a technical solution to overcome or at least alleviate at least one of the above-mentioned defects of the prior art. Summary of the invention

[0005] The purpose of this application is to provide a speed regulation method based on a video deep learning algorithm to solve at least one of the above-mentioned problems.

[0006] Glossary:

[0007] Variational autoencoder (VAE).

[0008] In a first aspect of the present application, a speed regulation method based on a video deep learning algorithm is provided, and the speed regulation method based on a video deep learning algorithm comprises:

[0009] Get image information of the conveyor belt and coal quantity;

[0010] Acquire the coal quantity classification information on the current transport belt according to the image information;

[0011] Determine whether the coal quantity classification information on the current conveyor belt matches the current conveyor belt conveyor speed. If not,

[0012] A speed regulation signal is generated according to the current coal quantity classification information on the transport belt and transmitted to the controller of the transport belt, and the controller changes the transport speed of the transport belt according to the speed regulation signal.

[0013] Optionally, obtaining the coal quantity classification information on the current transport belt according to the image information includes:

[0014] Get the trained coal quantity level classifier generated by VAE and CNN;

[0015] Obtaining image features from image information;

[0016] The image features are input into the coal quantity grade classifier to obtain coal quantity classification information, wherein the coal quantity classification information includes at least two coal quantity grades.

[0017] Optionally, obtaining the coal quantity classification information on the current transport belt according to the image information includes:

[0018] Get the trained coal quantity level classifier generated by VAE and GAN;

[0019] Obtaining image features from image information;

[0020] The image features are input into the coal quantity grade classifier to obtain coal quantity classification information, wherein the coal quantity classification information includes at least two coal quantity grades.

[0021] Optionally, the determining whether the coal quantity classification information on the current transport belt matches the current transport speed of the transport belt includes:

[0022] Acquire an associated database, wherein the associated database includes a plurality of coal quantity levels and transport belt transport speed interval information corresponding to each coal quantity level;

[0023] According to the obtained coal quantity level, the transport belt transport speed interval information corresponding to the coal quantity level in the associated database is obtained;

[0024] Get the current transport speed of the transport belt;

[0025] It is determined whether the current transport speed of the transport belt is within the acquired transport speed interval information of the transport belt.

[0026] Optionally, before the speed regulation signal is generated according to the coal quantity classification information on the current conveying belt and transmitted to the controller of the conveying belt, and the controller changes the conveying speed of the conveying belt according to the speed regulation signal, the speed regulation method based on the video deep learning algorithm further includes:

[0027] Acquire foreign body detection images;

[0028] Determine whether there is any foreign matter on the conveyor belt according to the foreign matter detection image; if so,

[0029] Generates a foreign object signal.

[0030] Optionally, judging whether there is a foreign object on the transport belt according to the foreign object detection image includes:

[0031] Get the trained foreign object recognition model generated by GAN and VAE;

[0032] Acquire foreign object image features in the foreign object detection image;

[0033] The foreign body image features are input into the foreign body recognition model to determine whether there is a foreign body.

[0034] The present application also provides a coal transport belt speed regulating device based on a video deep learning algorithm, and the coal transport belt speed regulating device based on a video deep learning algorithm comprises:

[0035] A coal quantity classification information acquisition module, wherein the coal quantity classification information acquisition module is used to obtain the coal quantity classification information on the current transport belt;

[0036] A judgment module, the judgment module is used to judge whether the coal quantity classification information on the current conveyor belt matches the current conveyor speed of the conveyor belt;

[0037] A speed control information generation module is used to generate a speed control signal according to the coal quantity classification information on the current transport belt and transmit it to the controller of the transport belt when the judgment module judges to be no, and the controller changes the transport speed of the transport belt according to the speed control signal.

[0038] The present application also provides a coal transportation system, the coal transportation system comprising:

[0039] A conveyor belt conveyor, wherein the conveyor belt conveyor is used for conveying coal;

[0040] A camera device, the camera device is used to photograph the conveyor belt and the coal on the conveyor belt, so as to obtain image information of the conveyor belt and the amount of coal;

[0041] A controller, the controller is connected to the conveyor belt conveyor and is used to control the conveying speed of the conveyor belt conveyor;

[0042] A coal transport belt speed regulating device based on video deep learning algorithm, wherein the coal transport belt speed regulating device based on video deep learning algorithm is connected to the camera device and the controller respectively, and is used to generate a speed regulating signal for the controller through the speed regulating method based on video deep learning algorithm as described above.

[0043] Optionally, the camera device further comprises acquiring a foreign body detection image;

[0044] The coal transport belt speed regulating device based on video deep learning algorithm further includes judging whether there are foreign objects on the transport belt according to the foreign object detection image, and if so, generating a foreign object signal and transmitting the foreign object signal to a controller, wherein the foreign object signal includes foreign object image features.

[0045] Optionally, the coal transportation system further comprises:

[0046] A plurality of foreign object position finding camera devices, each of which is arranged at intervals along the belt conveying direction of the conveying belt, and the distance between every two adjacent foreign object position finding camera devices is the same, and each of the foreign object position finding camera devices is connected to the controller respectively;

[0047] Each of the foreign object position finding camera devices performs the following operations:

[0048] Obtain the foreign object signal transmitted by the controller;

[0049] Acquire the image information that it can obtain;

[0050] Extracting image features of the image information acquired by itself;

[0051] Determine the similarity between the image features of the image information obtained by itself and the image features of the foreign object, and if the similarity is greater than a threshold, generate confirmation information and transmit it to the controller;

[0052] The controller performs the following operations:

[0053] Acquire identification information of the foreign object location-finding camera device that sends the confirmation information;

[0054] Acquire a position database, wherein the position database includes identification information of each foreign object position finding camera device and position information of each foreign object position finding camera device, wherein the position information of one foreign object position finding camera device corresponds to the identification information of one foreign object position finding camera device, and the position information of each foreign object position finding camera device includes the distance of the foreign object position finding camera device from a preset starting point;

[0055] According to the identification information of the camera device for finding the foreign object position and the position database, the position information of the camera device for finding the foreign object position is obtained;

[0056] Acquire preset stop position information, wherein the preset stop position information includes a distance between the preset stop position and the preset starting point;

[0057] Determine whether the distance between the foreign object position finding camera device and the preset starting point in the position information of the foreign object position finding camera device obtained is less than the distance between the preset stop position and the preset starting point; if not, then

[0058] Calculate the difference between the distance between the foreign object position finding camera device and the preset starting point and the distance between the preset stop position and the preset starting point in the position information of the foreign object position finding camera device, and the difference is called the waiting distance to be traveled;

[0059] Get the current transport speed of the transport belt;

[0060] Calculating an estimated time required to travel the distance to be traveled at the current transport speed of the transport belt according to the current transport speed of the transport belt and the distance to be traveled;

[0061] After the estimated time is calculated, a stop signal is sent to the transport belt after waiting for a time equal to the estimated time to stop the transport of the transport belt.

[0062] This application has at least the following beneficial technical effects:

[0063] The speed regulation method based on video deep learning algorithm of the present application classes the amount of coal on the conveyor belt by means of image recognition to obtain coal classification information, and controls the speed of the conveyor belt according to the coal classification information, so that the speed of the conveyor belt can always match the coal amount, thereby saving improper energy consumption. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is a flowchart of a speed regulation method based on a video deep learning algorithm provided by one embodiment of the present application;

[0065] Figure 2 An electronic device used to implement Figure 1 The speed regulation method based on video deep learning algorithm is shown.

[0066] Figure 3 yes Figure 1 A schematic diagram of images collected by a coal mine monitoring camera used for a coal quantity level classifier in a speed regulation method based on a video deep learning algorithm is shown;

[0067] Figure 4 yes Figure 1 The confusion matrix result diagram of the coal quantity level classifier of the speed regulation method based on the video deep learning algorithm and the method of this embodiment on the test set is shown;

[0068] Figure 5 yes Figure 1 The confusion matrix result diagram of the coal quantity level classifier of the speed regulation method based on the video deep learning algorithm and the method of this embodiment on the test set is shown;

[0069] Figure 6 yes Figure 1A schematic diagram of images collected by a coal mine monitoring camera used for a foreign object recognition model in a speed regulation method based on a video deep learning algorithm is shown;

[0070] Figure 7 yes Figure 1 A representative image diagram of normal image data and abnormal image data collected by a coal mine monitoring camera for a foreign object recognition model in a speed regulation method based on a video deep learning algorithm is shown;

[0071] Figure 8 It is a schematic diagram of the ROC curve based on the foreign body recognition model;

[0072] Fig. 9 This is the confusion matrix diagram of this application and other five baseline methods based on the foreign object recognition model;

[0073] Fig.10 It is the anomaly score histogram based on the foreign object recognition model on the test set. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical scheme and advantages of the implementation of this application clearer, the technical scheme in the embodiment of this application will be described in more detail below in conjunction with the drawings in the embodiment of this application. In the drawings, the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The described embodiments are part of the embodiments of this application, not all of them. The embodiments described below with reference to the drawings are exemplary and are intended to be used to explain this application, and should not be construed as limitations on this application. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application. The embodiments of this application are described in detail below in conjunction with the drawings.

[0075] It should be noted that, in the description of the present invention, the terms “first” and “second” are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0076] Figure 1 It is a flowchart of a speed regulation method based on a video deep learning algorithm provided in one embodiment of the present application.

[0077] like Figure 1 The speed regulation method based on the video deep learning algorithm shown includes:

[0078] Step 1: Obtain image information of the conveyor belt and coal quantity;

[0079] Step 2: Obtain the coal quantity classification information on the current transport belt according to the image information;

[0080] Step 3: Determine whether the coal quantity classification information on the current conveyor belt matches the conveyor speed of the current conveyor belt. If not,

[0081] Step 4: Generate a speed regulation signal according to the current coal quantity classification information on the transport belt and transmit it to the controller of the transport belt. The controller changes the transport speed of the transport belt according to the speed regulation signal.

[0082] The speed regulation method based on video deep learning algorithm of the present application classes the amount of coal on the conveyor belt by means of image recognition to obtain coal classification information, and controls the speed of the conveyor belt according to the coal classification information, so that the speed of the conveyor belt can always match the coal amount, thereby saving improper energy consumption.

[0083] In this embodiment, obtaining the coal quantity classification information on the current transport belt according to the image information includes:

[0084] Get the trained coal quantity level classifier generated by VAE and CNN;

[0085] Obtaining image features from image information;

[0086] The image features are input into the coal quantity grade classifier to obtain coal quantity classification information, wherein the coal quantity classification information includes at least two coal quantity grades.

[0087] In this embodiment, the variational auto-encoder (VAE) was first proposed by Kingma and Welling in 2014, which is equivalent to adding noise to the code on the basis of the encoding of the ordinary auto-encoder, so that the data of a set of noisy codes after the decoding process is as close as possible to the input data.

[0088] The variational autoencoder is built with two neural networks, forming two probability density distributions. One acts as an encoder to generate the average code and standard deviation, rather than directly generating the code, and then extracts the actual code from the Gaussian distribution of the average and standard deviation. The other acts as a decoder to decode, restore and reconstruct the code. The process of VAE generating data is generally shown as follows:

[0089]

[0090] Where: z is the latent variable, x is the sample, p(z) is the prior distribution, p(x|z) is the generated distribution of the sample under the condition of the given latent variable, which is often regarded as a decoder because it can complete the mapping from the latent space to the sample space.

[0091] The generative process of a conventional VAE consists of two steps. First, the latent variable z is sampled from a prior distribution, which is usually set to a standard Gaussian distribution. Then, samples are generated from the conditional distribution given the latent variable. The loss of VAE is a negative evidence lower bound (ELBO), which can be viewed as a negative log-likelihood function with a regularization term:

[0092]

[0093] Where: For expectation, D KL [·] is the KL divergence, q(z|x) is the posterior distribution of the latent variable under the condition of a given sample, which is often regarded as an encoder because it can complete the mapping from the sample space to the latent space.

[0094] The first term in the above formula is generally regarded as the reconstruction loss, that is, the mean square error between the input sample and the reconstructed sample. The second term is generally regarded as a regularization term, which is to make the posterior distribution closer to the prior distribution, where the prior distribution generally uses a standard Gaussian distribution. Therefore, the loss function of VAE can be rewritten as:

[0095]

[0096] In this embodiment, the trained coal quantity grade classifier generated by VAE and CNN includes two parts, one part is a variational autoencoder for reconstructing images, and the other part is a convolutional neural network for coal quantity classification.

[0097] In this example, VAE is used for the following reasons:

[0098] (1) Since VAE uses variational inference to simultaneously learn the mapping relationship between real image data and latent variables, it can capture the distribution of data. If the hierarchical prediction is to be sufficiently accurate, the important information in the data must be preserved, and a sufficiently accurate data distribution is also required. VAE can do this, which is the first reason why this paper chooses VAE to reconstruct images.

[0099] (2) Since the loss function of VAE is based on the reconstruction loss of mean square error, it is trained for each pixel of the image and cannot capture the global information of the image. Therefore, the image generated by VAE is blurred and lacks details. However, it is precisely because of this that the image reconstructed by VAE can remove a lot of unimportant noise information, making the image smoother and retaining only the more important image information. Therefore, using the reconstructed image to train CNN for belt coal grading prediction will not be disturbed by noise information. This is the second reason why this paper chooses VAE to reconstruct the original image.

[0100] In summary, by reconstructing images through VAE, we can retain only the important information of the image and remove the unimportant noise information, which is helpful for subsequent CNN training. In subsequent experiments, we will show the effect of reconstructed images and the accuracy difference between training CNN using original images and reconstructed images.

[0101] In this embodiment, after training the VAE, we will use the VAE to obtain the reconstructed data of the original image, and then use the reconstructed data of the original image to train a CNN for belt coal quantity classification prediction.

[0102] See also Figure 3 In this embodiment, the data used is collected from the coal mine through a camera and manually labeled. The coal quantity classification information includes four coal quantity levels, namely, A, B, C, and D. Among them, level A has the least coal quantity and level D has the most coal quantity. In order to facilitate subsequent experiments, the resolution of the image is adjusted to a 128*128 grayscale image. Figure 3 The following are representative diagrams of the four levels.

[0103] The obtained data was divided into a training set and a test set. The training set was used to train the model, and the test set was used to test the model. There were no duplicate samples in these two parts. The training set had 350 samples, and the sample size ratio of the four levels was 100:100:100:50. The test set had 178 samples, and the sample size ratio of the four levels was 50:50:50:28.

[0104] In order to make the model converge faster and obtain better experimental results and prediction accuracy, this paper preprocessed the selected samples and normalized all input samples:

[0105]

[0106] Where: x is the original image data, is the image data after normalization preprocessing, x minis the minimum pixel value in the original image, x max is the maximum pixel value in the original image.

[0107] See Table 1 below, which shows the network structure and hyperparameter settings of the VAE part in our method. In the encoder, a three-layer convolutional neural network is first used to better extract the feature information in the image. The hyperparameter settings of these convolutional layers (Conv2D) are as follows: the size of the convolution kernel is 6, the strides are 2, the padding is 2, the filters are 32, 64, and 128, and the ReLU function is used for activation; then three linear layers are used, where the first linear layer is to recombine the output information of the convolution layer, the number of neurons is set to 1024, and the ReLU function is used for activation. The second and third linear layers are to output the corresponding mean and variance in the VAE reparameterization technique, the number of neurons is 100, and no activation function is used. In the decoder, two linear layers are used first, with the number of neurons being 1024 and 16*16*128 respectively, and the ReLU function is used for activation; then three deconvolution layers (Conv2DTranspose) are used, with the convolution kernel size being 6, the step size being 2, the padding value being 2, the filters being 64, 32, and 1, and the Sigmoid function being used for activation in the last layer. When training VAE, the optimizer used is the RMSprop optimizer, with a learning rate of 0.0001, and batch training is used, with each batch of data being 32 (batch size), and a total of 50 epochs.

[0108] Table 1: VAE network structure

[0109]

[0110] See the table below. Table 2 shows the network structure and hyperparameter settings of the CNN part in our method. First, three convolutional layers were used, with the convolution kernel size of 6, the step size of 2, the padding value of 2, the number of filters of 32, 64, and 128, and the LeakyReLU function was used for activation. The negative semi-axis slope of the LeakyReLU function was set to 0.2. Then two linear layers were used, with the number of neurons being 1024 and 4, respectively, and the LeakyReLU function was used to activate the first linear layer. When training CNN, the optimizer used was the Adam optimizer, the learning rate was set to 0.001, and batch training was used, with each batch of 32 data, and a total of 100 iterations.

[0111] Table 2 CNN structure

[0112]

[0113] To demonstrate the superiority of our method, we compared it with several baseline methods, including four traditional machine learning algorithms, namely decision tree, K-nearest neighbors (KNN), random forest, and support vector machine (SVM), as well as two deep learning methods, deep neural network (DNN) and CNN. Among them, decision tree, K-nearest neighbors, random forest, and support vector machine were experimented using the sklearn machine learning library. DNN and CNN use the deep learning framework. In order to be able to compare with our method fairly, the CNN uses the same network structure and hyperparameter settings as the CNN components in our method.

[0114] For the evaluation indicators, four evaluation indicators were selected, namely accuracy, precision, recall and F1 score. At the same time, the classification effect of our method and CNN was compared through the confusion matrix. The above-mentioned evaluation indicators were calculated using the sklearn machine learning library.

[0115] See the table below. Table 3 shows the accuracy comparison between our method and the other six methods. By analyzing the above table, we can see that our method is better than the other six methods under the four evaluation indicators. Some traditional machine learning classification methods, such as decision tree, K nearest neighbor algorithm and random forest, cannot handle the problem of belt coal classification well. This is because traditional machine learning algorithms are not good at handling complex image processing problems in real scenes. Therefore, these three algorithms obtain lower values ​​under the four evaluation indicators. As for the traditional SVM algorithm, it does surpass the other three traditional machine learning algorithms, but its accuracy is still not as good as the deep learning method. For DNN, its accuracy is low because DNN uses linear layers, which are generally not suitable for processing image data. The accuracy of CNN is close to our method, but still not as good as our method. This is because our method first uses VAE to reconstruct the image, which can remove some unimportant noise information to avoid interfering with CNN training.

[0116] Table 3 Accuracy comparison

[0117]

[0118]

[0119] Figure 4 and Figure 5 The confusion matrix results of CNN and the method of this embodiment on the test set are given respectively. By analyzing the above figures, it can be seen that the accuracy of both methods in judging the C level is the highest, while the accuracy of judging the B level is lower. Figure 4 and Figure 5 It can also be seen that the classification accuracy of CNN is lower than that of our method, especially when judging level A and level B, while it is comparable to our method in judging level C and level D.

[0120] Figure 6 The following is the reconstruction effect of VAE on the original image. Figure 6 It can be found that the reconstruction effect can retain the important information of the original image, while making the image smoother and removing a lot of unimportant noise information. However, the reconstruction effect of level B is not very good, which can also explain why Figure 3 and Figure 4 The judgment accuracy of level B is lower than that of the other three levels.

[0121] This approach has the following advantages:

[0122] 1. The reconstructed image of the variational autoencoder can retain the important information of the original belt image data, and at the same time remove some noise information in the original image to make the image smoother.

[0123] 2. The convolutional neural network can be used to effectively classify the belt coal quantity data.

[0124] 3. The effect of using the reconstructed image of the variational autoencoder to train the grading algorithm is better than the grading prediction effect using the original image data.

[0125] In another embodiment, the present application obtains the coal quantity classification information on the current transport belt according to the image information, including:

[0126] Get the trained coal quantity level classifier generated by VAE and GAN;

[0127] Obtaining image features from image information;

[0128] The image features are input into the coal quantity grade classifier to obtain coal quantity classification information, wherein the coal quantity classification information includes at least two coal quantity grades.

[0129] In this alternative embodiment, the coal quantity grade classifier includes two parts, one is a generator network and the other is a discriminator network.

[0130] In this alternative embodiment, the generator network utilizes a variational auto-encoder (VAE). Specifically, for the generator network, the method in this paper does not use the generator network in the original GAN, but uses VAE as the generator network. This is because the purpose of this paper is to perform denoising, and the input of the model is no longer the noise sampled from the prior distribution, but a noisy image, so the generator network of the original GAN ​​is not suitable for the problem in this paper. The reason for choosing VAE as the generator network is that VAE can capture the distribution of the original data very well. At the same time, due to the characteristics of the VAE loss function, it will ignore a lot of noise information while learning the data distribution, which is beneficial to the denoising task.

[0131] In this alternative embodiment, the discriminator network uses the discriminator network of the original GAN. The purpose of introducing the discriminator network is that the images generated by VAE are often blurry and lack details, while GAN can learn the details of the image very well, so the introduction of the discriminator can obtain a clearer image. As for the input of the discriminator, what is different in this article is that its input images are the original coal mine monitoring images and the coal mine monitoring images reconstructed by VAE, rather than the false images generated by the generator.

[0132] In this alternative embodiment, the loss function of this paper consists of two parts, one of which is the VAE loss, which is used to reconstruct the image, and the other is the adversarial loss, which is used to improve the image clarity. The total loss function of the method in this paper is shown as follows:

[0133]

[0134] Where: represents the VAE loss, represents the adversarial loss and λ represents the hyperparameter.

[0135] For VAE loss, the method in this paper uses the same loss function as the original VAE

[16] . The VAE loss function is a negative evidence lower bound (ELBO) derived from the likelihood function. It can be regarded as a reconstruction loss with a regularization term, which consists of two parts: the first term is the reconstruction loss, that is, the mean square error between the original input image and the reconstructed output image; the second term is the regularization term. In general, the prior distribution chooses the standard normal distribution. The VAE loss is shown as follows:

[0136]

[0137] Where: x represents the original image, represents the reconstructed image, DKL [·] represents KL divergence (Kullback-Leibler divergence), q φ (z|x) represents the posterior distribution of z given x, Represents a standard Gaussian distribution.

[0138] For adversarial loss, this paper makes some modifications based on the original adversarial loss of GAN. Since the discriminator input of this paper's method is the original image and the image reconstructed by VAE, rather than the image generated by noise, the adversarial loss needs to be modified. The modified adversarial loss is shown in the following formula:

[0139]

[0140] Where: G represents the generator (ie VAE), and D represents the discriminator.

[0141] In this alternative embodiment, the experiment uses two data sets, which are image data collected by two different coal mine monitoring cameras in the coal mine. The two data sets are named S1 and S2. In order to facilitate the subsequent experiments, all images are adjusted to grayscale images with a pixel value of 256*512. The representative image data of the two data sets are as follows: Figure 6 shown.

[0142] In this embodiment, judging whether the coal quantity classification information on the current transport belt matches the transport speed of the current transport belt includes:

[0143] Acquire a related database, the related database including a plurality of coal quantity levels and transport belt transport speed interval information corresponding to each coal quantity level;

[0144] According to the obtained coal quantity level, the transport belt transport speed interval information corresponding to the coal quantity level in the associated database is obtained;

[0145] Get the current transport speed of the transport belt;

[0146] It is determined whether the current conveying speed of the conveying belt is within the obtained conveying speed interval information of the conveying belt.

[0147] For example, as mentioned above, the coal quantity level is 4, that is, Figure 3There are four levels of A, B, C, and D in the belt conveyor, and each level sets a conveyor belt conveyor speed interval information, that is, the conveyor belt conveyor speed interval information also includes four numerical ranges, for example, 2 m / min to 3 m / min; 4 m / min to 5 m / min, 6 m / min to 7 m / min, 8 m / min to 9 m / min; among them, 2 m / min to 3 m / min corresponds to level A, 4 m / min to 5 m / min corresponds to level B; 6 m / min to 7 m / min corresponds to level C; 8 m / min to 9 m / min corresponds to level D.

[0148] According to the obtained coal quantity level, the transport belt transport speed interval information corresponding to the coal quantity level in the associated database is obtained. For example, if the obtained coal quantity level is A, it corresponds to 2 m / min to 3 m / min.

[0149] Obtain the current transport speed of the transport belt (obtaining the transport speed of the transport belt is a prior art and will not be described in detail here), and determine whether the current transport speed of the transport belt is within the acquired transport speed interval information of the transport belt. For example, if the current transport speed of the transport belt after acquisition is 2.5 m / min, then it is within 2 m / min to 3 m / min. If the acquired current transport speed of the transport belt is 5 m / min, it does not correspond to the coal quantity grade A, that is, it does not match. Then, a speed control signal is generated according to the coal quantity classification information on the current transport belt and transmitted to the controller of the transport belt. The controller changes the transport speed of the transport belt according to the speed control signal, so that the current transport speed of the transport belt is within 2 m / min to 3 m / min.

[0150] In this embodiment, before a speed regulation signal is generated according to the coal quantity classification information on the current conveying belt and transmitted to the controller of the conveying belt, and the controller changes the conveying speed of the conveying belt according to the speed regulation signal, the speed regulation method based on the video deep learning algorithm further includes:

[0151] Acquire foreign body detection images;

[0152] Determine whether there is any foreign matter on the conveyor belt according to the foreign matter detection image; if so,

[0153] Generates a foreign object signal.

[0154] In this embodiment, judging whether there is a foreign object on the conveying belt according to the foreign object detection image includes:

[0155] Get the trained foreign object recognition model generated by GAN and VAE;

[0156] Acquire foreign object image features in the foreign object detection image;

[0157] The foreign body image features are input into the foreign body recognition model to determine whether there is a foreign body.

[0158] In this embodiment, since the abnormal objects on the belt coal mine conveyor are usually mixed with coal, the foreign body recognition model used in this application is generated by GAN and VAE. Specifically, since the loss of VAE is a reconstruction loss based on the mean square error, the reconstruction error can be used to determine whether there are abnormal objects in a picture, so VAE is a very suitable method for abnormality detection. However, since the abnormal objects on the belt coal mine conveyor are usually mixed with coal, and the reconstructed image of VAE will ignore some detailed information, it is not applicable to the detection of foreign objects on the belt. GAN can capture a lot of detailed information in the image, so we combine GAN with VAE to make it suitable for foreign body detection in coal mine belt conveyors.

[0159] In this embodiment, GAN is a deep generative model in which the training process is in a game state. The main structure of GAN can be divided into two parts: the generator network and the discriminator network. The function of the generator network is to generate false images that are visually similar to real images, while the role of the discriminator is to determine whether the input signal comes from the real data set or from the false image generated by the generator.

[0160] During the training process, a set of noise is sampled from a prior distribution (generally a Gaussian distribution), and then passed through the generator to generate a set of false images. The generated false images are then input into the discriminator together with the real images collected from the dataset to determine where the input images come from. After several rounds of iterations, the images generated by the generator can deceive the discriminator, and GAN reaches the optimal state. The objective function of GAN is as follows:

[0161]

[0162] Where: G is the generator and D is the discriminator.

[0163] The foreign body recognition model of the present application includes three parts, namely an encoder, a decoder and a discriminator. The encoder and the decoder together constitute a VAE, and the VAE is regarded as a generator, which together with the discriminator constitutes a GAN.

[0164] The role of the encoder is to encode the high-dimensional image and obtain a compressed low-dimensional representation. The encoder completes the mapping from the high-dimensional original image space to the latent space. At the output of the encoder, the reparameterization technique is used. The role of the decoder is to decode the low-dimensional representation and obtain the decoded reconstructed image. The decoder completes the mapping from the low-dimensional latent space to the high-dimensional reconstructed space. The encoder and decoder together constitute a VAE, and this VAE is regarded as the generator part of the GAN.

[0165] The role of the discriminator is to determine whether the input image is a real image or a generated image. In this method, since VAE is used as the generator part of GAN, this generator does not sample noise from a Gaussian distribution to generate an image, but reconstructs the original image. Therefore, the role of this discriminator is to determine whether the input image comes from a real image or a reconstructed image.

[0166] In this embodiment, the loss function is composed of two parts, namely VAE loss and adversarial loss. The total loss function is shown as follows:

[0167]

[0168] Where: is the VAE loss, For adversarial loss, λ is a hyperparameter that balances the impact of the two sub-losses on the total loss.

[0169] For VAE loss, the method in this paper directly uses the loss function in VAE, as shown in the following formula. VAE loss can be regarded as a reconstruction loss with a regularization term, which consists of two parts: the first term is the reconstruction loss, that is, the mean square error between the original image and the reconstructed image, which is to make the reconstructed image as similar to the original image as possible; the second term is the regularization term, which is to make the posterior distribution closer to the prior distribution. In general, the prior distribution chooses the standard Gaussian distribution.

[0170]

[0171] Where: x is the original image, To reconstruct the image, D KL [·] is the KL divergence, q φ (z|x) is the posterior distribution of z given x, is a standard Gaussian distribution.

[0172] For the adversarial loss, this paper makes some modifications based on the original adversarial loss of GAN. Since the generator of this method does not generate images by sampling noise from a Gaussian distribution, but uses VAE as a generator to reconstruct the original image, the input of the generator is no longer noise but the original image. The modified adversarial loss is shown in the following formula:

[0173]

[0174] The anomaly score can detect whether there are foreign objects in the image. The higher the anomaly score, the more likely it is that there are foreign objects, and the lower the anomaly score, the less likely it is that there are foreign objects. In this method, the anomaly score used is the reconstruction error between the test image and the reconstructed image of the test image:

[0175]

[0176] Where: x test For the test image, is the reconstructed image of the test image.

[0177] When the anomaly score is greater than a certain threshold, it is considered that a foreign object appears in the image, otherwise no foreign object appears:

[0178]

[0179] Where: τ is the threshold.

[0180] For the selection of threshold, in this paper, the quantile of the anomaly ratio is used as the threshold.

[0181] In this embodiment, the data set is collected from a coal mine through a camera, and the collected image data is manually annotated. The data is divided into two categories: one is normal image data without foreign matter, and the other is abnormal image data with foreign matter. In order to facilitate subsequent experiments, all image data are adjusted to grayscale images with a pixel value of 256*256. Representative data of normal image data and abnormal image data are as follows: Figure 7 shown.

[0182] The anomaly detection algorithm and baseline method used in this application are both semi-supervised learning algorithms. Therefore, when dividing the training set and the test set, the training set contains only normal samples, including 800 normal images. In order to be able to evaluate the differences between the algorithms more fairly, the sample size of normal samples and abnormal samples in the test set is the same, 100 each. The specific information of the training set and the test set is shown in Table 4.

[0183] Table 4 Dataset information

[0184] Dataset Pixel Normal sample Abnormal samples Abnormal ratio Training set 256*256 800 0 0％ Test Set 256*256 100 100 50％

[0185] In order to train the model more stably and obtain better experimental results, all image data were normalized and preprocessed:

[0186]

[0187] In the formula: x represents the original data, represents the data after normalization preprocessing, x min Represents the minimum pixel value in the image, x max Represents the maximum pixel value in the image.

[0188] In this embodiment, the foreign body recognition method of the present application is compared with five baseline methods, including two traditional machine learning algorithms: One-Class Support Vector Machine (OCSVM) and Kernel Density Estimation (KDE); three deep learning methods: Auto-Encoder (AE), Denoising Auto-Encoder (DAE) and Variational Auto-Encoder (VAE);

[0189] For the network structure setting, the network structure of this method is as follows Figure 1 In order to compare the methods more fairly, the components in AE, DAE, and VAE are the same as those in this paper. The two traditional machine learning methods, OCSVM and KDE, use the default parameters in the machine learning library sklearn.

[0190] For the evaluation indicators, this paper selected two evaluation indicators for classification tasks, namely F1 score and AUC (Area Under the Curve). At the same time, ROC curve and confusion matrix were used to compare the classification effects of the six methods.

[0191] Table 5 shows the accuracy comparison results of the proposed method and the other five baseline methods. The table shows the F1 score and AUC results corresponding to each method. By analyzing Table 5, it can be seen that the effects of the two classic machine learning methods are not as good as the deep learning method, and the accuracy of KDE is poor. This is because traditional machine learning methods have difficulties in processing high-dimensional and complex real scene image tasks, and may suffer from the dimensionality curse problem when applied. The accuracy performance of the three deep learning methods, AE, DAE, and VAE, is relatively similar, but their accuracy performance is still not as good as that of the proposed method. This is because the reconstructed images of AE, DAE, and VAE are often blurred and lack details, which is difficult for the real scene coal mine foreign body detection task, because the foreign matter in the coal mine may only occupy a small position in the collected image. The proposed method uses GAN to learn the detailed information of the image, so as to achieve better foreign body detection effect.

[0192] Table 5 Accuracy comparison

[0193]

[0194]

[0195] Figure 8 The ROC curves of the proposed method and the other five baseline methods are given in the figure. Figure 8 It can be seen intuitively that our method has better classification effect than other baseline methods, and the deep learning method is superior to the traditional memory learning method.

[0196] Fig. 9 The following is a confusion matrix diagram of this application and five other baseline methods. Fig. 9 It can be seen that the foreign matter detection effect of this application is the best.

[0197] Fig.10 The following is a histogram of the anomaly scores of the proposed method on the test set. We show the scores of normal test samples and abnormal test samples respectively. Fig.10 It can be seen that the scores of normal test samples are significantly lower than those of abnormal test samples, which shows that the proposed method can clearly distinguish normal test samples from abnormal test samples, that is, it can accurately detect foreign objects. At the same time, it can be found that the score dispersion of normal test samples is smaller than that of abnormal test samples, which shows that the proposed method can capture the distribution of normal samples and accurately distinguish samples outside the distribution of normal samples.

[0198] The foreign body detection method of the present application has the following advantages:

[0199] (1) VAE can reconstruct the belt image and use the reconstruction error to determine whether there is foreign matter on the belt.

[0200] (2) GAN can improve the quality of VAE’s reconstructed images, enabling it to learn some detailed information in the image so as to achieve better foreign body detection results.

[0201] (3) The anomaly scores obtained by the proposed method can separate normal samples from abnormal samples, and the discrete degree of normal sample scores is smaller than that of abnormal sample scores.

[0202] The present application also provides a coal transport belt speed control device based on video deep learning algorithm, and the coal transport belt speed control device based on video deep learning algorithm includes a coal quantity grading information acquisition module, a judgment module and a speed control information generation module. The coal quantity grading information acquisition module is used to obtain the coal quantity grading information on the current transport belt; the judgment module is used to judge whether the coal quantity grading information on the current transport belt matches the transport speed of the current transport belt; the speed control information generation module is used to generate a speed control signal according to the coal quantity grading information on the current transport belt when the judgment module judges to be no, and transmit it to the controller of the transport belt, and the controller changes the transport speed of the transport belt according to the speed control signal.

[0203] The present application also provides a coal transportation system, which includes a conveyor belt conveyor, a camera device, a controller, and a coal transportation belt speed regulating device based on a video deep learning algorithm, wherein:

[0204] Used to transport coal on conveyor belts;

[0205] The camera device is used to photograph the conveyor belt and the coal on the conveyor belt, so as to obtain image information of the conveyor belt and the amount of coal;

[0206] The controller is connected to the conveyor belt conveyor and is used to control the conveying speed of the conveyor belt conveyor;

[0207] The coal transport belt speed regulating device based on video deep learning algorithm is respectively connected to the camera device and the controller, and is used to generate a speed regulating signal for the controller through the speed regulating method based on video deep learning algorithm as described above.

[0208] In this embodiment, the camera device further includes acquiring a foreign body detection image;

[0209] The coal transport belt speed regulating device based on video deep learning algorithm further includes judging whether there are foreign objects on the transport belt according to the foreign object detection image, and if so, generating a foreign object signal and transmitting the foreign object signal to a controller, wherein the foreign object signal includes foreign object image features.

[0210] In this embodiment, the coal transportation system further includes:

[0211] A plurality of foreign object position finding camera devices, each of which is arranged at intervals along the belt conveying direction of the conveying belt, and the distance between every two adjacent foreign object position finding camera devices is the same, and each of the foreign object position finding camera devices is connected to the controller respectively;

[0212] Each of the foreign object position finding camera devices performs the following operations:

[0213] Obtain the foreign object signal transmitted by the controller;

[0214] Acquire the image information that it can obtain;

[0215] Extracting image features of the image information acquired by itself;

[0216] Determine the similarity between the image features of the image information obtained by itself and the image features of the foreign object, and if the similarity is greater than a threshold, generate confirmation information and transmit it to the controller;

[0217] The controller performs the following operations:

[0218] Acquire identification information of the foreign object location-finding camera device that sends the confirmation information;

[0219] Acquire a position database, the position database includes identification information of each foreign object position finding camera device and position information of each foreign object position finding camera device, the position information of one foreign object position finding camera device corresponds to the identification information of one foreign object position finding camera device, and the position information of each foreign object position finding camera device includes the distance between the foreign object position finding camera device and a preset starting point;

[0220] According to the identification information of the camera device for finding the foreign object position and the position database, the position information of the camera device for finding the foreign object position is obtained;

[0221] Acquire preset stop position information, wherein the preset stop position information includes a distance between the preset stop position and the preset starting point;

[0222] Determine whether the distance between the foreign object position finding camera device and the preset starting point in the position information of the foreign object position finding camera device obtained is less than the distance between the preset stop position and the preset starting point; if not, then

[0223] Calculate the difference between the distance between the foreign object position finding camera device and the preset starting point and the distance between the preset stop position and the preset starting point in the position information of the foreign object position finding camera device, and the difference is called the waiting distance to be traveled;

[0224] Get the current transport speed of the transport belt;

[0225] Calculating an estimated time required to travel the distance to be traveled at the current transport speed of the transport belt according to the current transport speed of the transport belt and the distance to be traveled;

[0226] After the estimated time is calculated, a stop signal is sent to the transport belt after waiting for a time equal to the estimated time to stop the transport of the transport belt.

[0227] In actual work, the conveyor belt may be relatively long, and the environment around the conveyor belt may be relatively complex. When foreign matter actually appears, on the one hand, if the foreign matter is not handled in time, it may be dangerous. On the other hand, even if the conveyor belt is stopped, it may be difficult for the user to clean up the foreign matter due to the location of the foreign matter. Through the above method, the foreign matter can follow the conveyor belt to a designated position, and the designated position can be set to a position that is convenient for the user to work. In this way, no matter where the foreign matter is, the above method can be used to make the foreign matter reach the predetermined position, making it convenient for the user to clean up the foreign matter.

[0228] For example, by shooting with a foreign object location finding camera device, the image taken by each foreign object location finding camera device can be obtained, and then the same processing method as the image feature processing camera device's image information of the conveyor belt and the amount of coal can be used to obtain the image features of the image taken by the foreign object location finding camera device.

[0229] The image features of the image taken by the foreign object location finding camera device are compared with the image features of the camera device obtained previously for similarity. If the similarity is greater than a threshold, it is considered that the shooting angle and the shooting position are close or close, so that it is possible to obtain which specific foreign object location finding camera device is closer to the foreign object, so that the foreign object location finding camera device that is closer to the foreign object generates confirmation information and sends it to the controller.

[0230] The controller obtains the identification information of the foreign object location finding camera device that sends the confirmation information, and obtains the location information of the foreign object location finding camera device based on the identification information of the foreign object location finding camera device and the location database. For example, the location information of the foreign object location finding camera device is 10 meters away from a preset starting point.

[0231] The preset stop position information is obtained, where the preset stop position information includes the distance between the preset stop position and the preset starting point. For example, the distance between the preset stop position and the preset starting point is 100 meters.

[0232] Determine whether the distance between the foreign object position finding camera device and the preset starting point in the position information of the foreign object position finding camera device is less than the distance between the preset stop position and the preset starting point, and if 10 meters is less than 100 meters, then

[0233] The difference between the distance from the foreign object position finding camera device to the preset starting point and the distance from the preset stop position to the preset starting point in the position information of the foreign object position finding camera device obtained is calculated. The difference is called the distance to be walked. In this example, the value is 90 meters.

[0234] Get the current conveyor belt speed. For example, if the current conveyor belt speed is 5 m / s, then

[0235] According to the current transport speed of the transport belt and the distance to be traveled, the estimated time required to travel the distance to be traveled at the current transport speed of the transport belt is calculated, that is, it takes 18 seconds to travel 90 meters.

[0236] After calculating the estimated time, a stop signal is sent to the conveyor belt after waiting for the same time as the estimated time to stop the conveyor belt, that is, it stops after waiting for 18 seconds. The foreign matter will be located near the preset stop position, which is convenient for the user to clean.

[0237] The present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the speed regulation method based on the video deep learning algorithm as described above when executing the computer program.

[0238] The present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the speed regulation method based on the video deep learning algorithm as described above.

[0239] Figure 2 It is an exemplary structural diagram of an electronic device that can implement the speed regulation method based on the video deep learning algorithm provided according to an embodiment of the present application.

[0240] like Figure 2As shown, the electronic device includes an input device 501, an input interface 502, a central processor 503, a memory 504, an output interface 505, and an output device 506. The input interface 502, the central processor 503, the memory 504, and the output interface 505 are interconnected through a bus 507, and the input device 501 and the output device 506 are connected to the bus 507 through the input interface 502 and the output interface 505, respectively, and then connected to other components of the electronic device. Specifically, the input device 504 receives input information from the outside, and transmits the input information to the central processor 503 through the input interface 502; the central processor 503 processes the input information based on the computer executable instructions stored in the memory 504 to generate output information, temporarily or permanently stores the output information in the memory 504, and then transmits the output information to the output device 506 through the output interface 505; the output device 506 outputs the output information to the outside of the electronic device for use by the user.

[0241] That is to say, Figure 2 The electronic device shown can also be implemented as comprising: a memory storing computer executable instructions; and one or more processors, which can implement the combination when executing the computer executable instructions. Figure 1 Described is a speed regulation method based on video deep learning algorithm.

[0242] In one embodiment, Figure 2 The electronic device shown can be implemented to include: a memory 504, configured to store executable program code; one or more processors 503, configured to run the executable program code stored in the memory 504 to execute the speed regulation method based on the video deep learning algorithm in the above embodiment.

[0243] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0244] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0245] Computer readable media include permanent and non-permanent, removable and non-removable, and the media can be implemented by any method or technology to store information. The information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), data versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0246] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0247] In addition, it is clear that the word "comprising" does not exclude other units or steps. Multiple units, modules or devices stated in the device claim can also be implemented by one unit or the overall device through software or hardware. The words first, second, etc. are used to identify the name, not any specific order.

[0248] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and a module, a program segment or a part of a code includes one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes identified in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or the total flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0249] The processor referred to in this embodiment may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0250] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the device / terminal equipment by running or executing the computer programs and / or modules stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0251] In this embodiment, if the module / unit integrated in the device / terminal equipment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0252] Finally, it should be pointed out that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them. Although the present invention has been described in detail with reference to the above embodiments, a person skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A speed regulation method based on video deep learning algorithm, characterized in that: The speed regulation method based on the video deep learning algorithm includes: Get image information of the conveyor belt and coal quantity; Acquire the coal quantity classification information on the current transport belt according to the image information; Determine whether the coal quantity classification information on the current conveyor belt matches the current conveyor belt conveyor speed. If not, Generate a speed regulation signal according to the current coal quantity classification information on the transport belt and transmit it to the controller of the transport belt, and the controller changes the transport speed of the transport belt according to the speed regulation signal; The step of obtaining the coal quantity classification information on the current transport belt according to the image information comprises: Get the trained coal quantity level classifier generated by VAE and CNN; Obtaining image features from image information; Inputting the image features into the coal quantity grade classifier to obtain coal quantity classification information, wherein the coal quantity classification information includes at least two coal quantity grades; The determining whether the coal quantity classification information on the current transport belt matches the current transport speed of the transport belt includes: Acquire an associated database, wherein the associated database includes a plurality of coal quantity levels and transport belt transport speed interval information corresponding to each coal quantity level; According to the obtained coal quantity level, the transport belt transport speed interval information corresponding to the coal quantity level in the associated database is obtained; Get the current transport speed of the transport belt; It is determined whether the current transport speed of the transport belt is within the acquired transport speed interval information of the transport belt.

2. The speed regulation method based on video deep learning algorithm according to claim 1, characterized in that: Before the speed regulation signal is generated according to the coal quantity classification information on the current conveying belt and transmitted to the controller of the conveying belt, and the controller changes the conveying speed of the conveying belt according to the speed regulation signal, the speed regulation method based on the video deep learning algorithm further includes: Acquire foreign body detection images; Determine whether there is any foreign matter on the conveyor belt according to the foreign matter detection image; if so, Generates a foreign object signal.

3. The speed regulation method based on video deep learning algorithm as claimed in claim 2, characterized in that: The determining whether there is a foreign object on the conveying belt according to the foreign object detection image comprises: Get the trained foreign object recognition model generated by GAN and VAE; Acquire foreign object image features in the foreign object detection image; The foreign body image features are input into the foreign body recognition model to determine whether there is a foreign body.

4. A coal transport belt speed regulating device based on video deep learning algorithm, characterized in that: The coal transport belt speed regulating device based on video deep learning algorithm includes: A coal quantity classification information acquisition module, wherein the coal quantity classification information acquisition module is used to obtain the coal quantity classification information on the current transport belt; A judgment module, the judgment module is used to judge whether the coal quantity classification information on the current conveyor belt matches the current conveyor speed of the conveyor belt; A speed regulation information generating module, wherein the speed regulation information generating module is used to generate a speed regulation signal according to the coal quantity classification information on the current transport belt and transmit the signal to the controller of the transport belt when the judgment module judges that the signal is no, and the controller changes the transport speed of the transport belt according to the speed regulation signal; The obtaining of the coal quantity classification information on the current transport belt includes: Get image information of the conveyor belt and coal quantity; Get the trained coal quantity level classifier generated by VAE and CNN; Obtaining image features from image information; Inputting the image features into the coal quantity grade classifier to obtain coal quantity classification information, wherein the coal quantity classification information includes at least two coal quantity grades; The determining whether the coal quantity classification information on the current transport belt matches the current transport speed of the transport belt includes: Acquire an associated database, wherein the associated database includes a plurality of coal quantity levels and transport belt transport speed interval information corresponding to each coal quantity level; According to the obtained coal quantity level, the transport belt transport speed interval information corresponding to the coal quantity level in the associated database is obtained; Get the current transport speed of the transport belt; It is determined whether the current transport speed of the transport belt is within the acquired transport speed interval information of the transport belt.

5. A coal transportation system, characterized in that: The coal transportation system comprises: A conveyor belt conveyor, wherein the conveyor belt conveyor is used for conveying coal; A camera device, the camera device is used to photograph the conveyor belt and the coal on the conveyor belt, so as to obtain image information of the conveyor belt and the amount of coal; A controller, the controller is connected to the conveyor belt conveyor and is used to control the conveying speed of the conveyor belt conveyor; A coal transport belt speed regulating device based on video deep learning algorithm, wherein the coal transport belt speed regulating device based on video deep learning algorithm is respectively connected to the camera device and the controller, and is used to generate a speed regulating signal for the controller through the speed regulating method based on video deep learning algorithm as described in any one of claims 1 to 3.

6. The coal transportation system according to claim 5, characterized in that: The camera device further comprises acquiring a foreign body detection image; The coal transport belt speed regulating device based on video deep learning algorithm further includes judging whether there are foreign objects on the transport belt according to the foreign object detection image, and if so, generating a foreign object signal and transmitting the foreign object signal to a controller, wherein the foreign object signal includes foreign object image features.

7. The coal transportation system according to claim 6, characterized in that: The coal transportation system further comprises: A plurality of foreign object position finding camera devices, each of which is arranged at intervals along the belt conveying direction of the conveying belt, and the distance between every two adjacent foreign object position finding camera devices is the same, and each of the foreign object position finding camera devices is connected to the controller respectively; Each of the foreign object position finding camera devices performs the following operations: Obtain the foreign object signal transmitted by the controller; Acquire the image information that it can obtain; Extracting image features of the image information acquired by itself; Determine the similarity between the image features of the image information obtained by itself and the image features of the foreign object, and if the similarity is greater than a threshold, generate confirmation information and transmit it to the controller; The controller performs the following operations: Acquire identification information of the foreign object location-finding camera device that sends the confirmation information; Acquire a position database, wherein the position database includes identification information of each foreign object position finding camera device and position information of each foreign object position finding camera device, wherein the position information of one foreign object position finding camera device corresponds to the identification information of one foreign object position finding camera device, and the position information of each foreign object position finding camera device includes the distance of the foreign object position finding camera device from a preset starting point; According to the identification information of the camera device for finding the foreign object position and the position database, the position information of the camera device for finding the foreign object position is obtained; Acquire preset stop position information, wherein the preset stop position information includes a distance between the preset stop position and the preset starting point; Determine whether the distance between the foreign object position finding camera device and the preset starting point in the position information of the foreign object position finding camera device obtained is less than the distance between the preset stop position and the preset starting point; if not, then Calculate the difference between the distance between the foreign object position finding camera device and the preset starting point and the distance between the preset stop position and the preset starting point in the position information of the foreign object position finding camera device, and the difference is called the waiting distance to be traveled; Get the current transport speed of the transport belt; Calculating an estimated time required to travel the distance to be traveled at the current transport speed of the transport belt according to the current transport speed of the transport belt and the distance to be traveled; After the estimated time is calculated, a stop signal is sent to the transport belt after waiting for a time equal to the estimated time to stop the transport of the transport belt.

Images