Detection Method, Device and Electronic Equipment for Conveyor Belt

Abstract

The present application provides a detection method, device and electronic device for a conveyor belt. The method includes: obtaining a historical image of the conveyor belt; performing compression processing and restoration processing on the historical image to obtain a historical reconstructed image, and constructing a reconstruction model based on the historical image and the historical reconstructed image; obtaining a current image of the conveyor belt, inputting the current image into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image; comparing the similarity between the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity. In this solution, a detection solution for a conveyor belt without abnormal samples for training is proposed, which can realize abnormal detection work by obtaining images of the conveyor belt working normally, without abnormal samples participating in the training model. Since samples of the conveyor belt in a normal state can be obtained and are also achievable, the accuracy of conveyor belt detection can be guaranteed to be relatively high.

Description

Technical Field

[0001] The present application relates to the field of coal conveyor belt detection, and specifically, to a conveyor belt detection method, device, computer-readable storage medium and electronic device. Background Art

[0002] The coal industry is a vital foundational industry in my country, and its sustainable development is crucial to the healthy development of the national economy and national energy security. During coal mining, problems such as conveyor belt deviation and large foreign objects entering the conveyor belt causing it to tear can sometimes occur, leading to safety accidents and significant economic losses.

[0003] Currently, the main methods for detecting conveyor belt anomalies include manual inspection, radiographic inspection, and video image inspection. Manual inspection requires dedicated workers to monitor the conveyor belt around the clock for abnormalities, resulting in high labor costs. Radiographic inspection requires specialized equipment to identify foreign objects, requiring high investment and maintenance costs, and cannot address conveyor belt deviations. Video image inspection leverages computer vision knowledge and uses conveyor belt videos captured by cameras to monitor the conveyor belt in real time. It automatically sends alarms when an anomaly is encountered, requiring relatively low labor and economic investment costs, making it the most widely used conveyor belt anomaly detection method.

[0004] Recognition methods based on video image detection primarily include manually designed image features and deep learning-based automatic feature extraction. Due to limitations such as design complexity associated with manual feature extraction, deep learning-based automatic feature extraction is currently more widely used. However, during coal mining, conveyor belt anomalies can cause significant economic losses. Therefore, maintaining the conveyor belt in an abnormal state in order to collect anomaly samples for model training is unrealistic. Consequently, it is currently difficult to collect anomaly samples from conveyor belts in an abnormal state, resulting in low conveyor belt detection accuracy. Summary of the Invention

[0005] The main purpose of this application is to provide a conveyor belt detection method, device, computer-readable storage medium and electronic device to solve the problem in the prior art that it is difficult to collect abnormal samples when the conveyor belt is in an abnormal state, resulting in low conveyor belt detection accuracy.

[0006] According to one aspect of the embodiments of the present invention, a detection method for a conveyor belt is provided, including: obtaining a historical image of the conveyor belt, where the historical image is collected by an image acquisition device; performing compression processing and restoration processing on the historical image to obtain a historical reconstructed image, and constructing a reconstruction model based on the historical image and the historical reconstructed image; obtaining a current image of the conveyor belt, inputting the current image into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image; comparing the similarity between the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity.

[0007] Optionally, in the process of constructing the reconstruction model based on the historical image and the historical reconstructed image, the method further includes: constructing an encoder module and a decoder module, where the encoder module is composed of a first number of convolutional neural network layers, and the decoder module is composed of a second number of convolutional neural network layers. The encoder module is used to compress the historical image of the first size into the historical image of the second size, and the decoder module is used to restore the historical image of the second size to the historical reconstructed image of the first size, where the first size is larger than the second size; obtaining first mapping information of the encoder module and second mapping information of the decoder module. The first mapping information refers to the mapping relationship between the historical image of the first size and the historical image of the second size, and the second mapping information refers to the mapping relationship between the historical image of the second size and the historical reconstructed image of the first size; determining the loss function of the reconstruction model according to the historical image, the first mapping information, and the second mapping information.

[0008] Optionally, after determining the loss function of the reconstruction model according to the historical image, the first mapping information, and the second mapping information, the method further includes: obtaining a plurality of first loss functions, where the first loss function is the loss function corresponding to the first mapping information, and the plurality of first loss functions are obtained by inputting a third number of the historical images as a training set into the reconstruction model; obtaining the average value of a plurality of second loss functions, where the second loss function is the loss function corresponding to the second mapping information, and the plurality of second loss functions are obtained by inputting a fourth number of the historical images as a validation set into the reconstruction model; determining the first loss function with the smallest difference from the average value as the target loss function, and determining the first mapping information corresponding to the target loss function as the target mapping information.

[0009] Optionally, determining whether the conveyor belt is abnormal according to the similarity includes: when the similarity is greater than or equal to a similarity threshold, determining that the conveyor belt is normal; when the similarity is less than the similarity threshold, obtaining a first area and a second area in the current image, where the first area refers to the area where the conveyor belt is located, and the second area refers to the area where the conveyor belt is not located; determining whether the conveyor belt is abnormal according to the position of the target area, where the target area refers to the area where the similarity between the current image and the current reconstructed image is less than the similarity threshold.

[0010] Optionally, obtaining the first area and the second area in the current image includes: obtaining a plurality of predetermined coordinate points, where the predetermined coordinate points are the position coordinate points in the current image for marking the first area; determining the area enclosed by the plurality of predetermined coordinate points as the first area; determining the area in the current image other than the first area as the second area.

[0011] Optionally, determining whether the conveyor belt is abnormal according to the position of the target area includes: when the target area is within the first area, determining that the conveyor belt is abnormal; when the target area is within the second area, determining that the conveyor belt is normal; when a part of the target area is within the first area and a part is within the second area, determining that the conveyor belt is abnormal.

[0012] Optionally, after determining whether the conveyor belt is abnormal according to the similarity, the method further includes: when it is determined that the conveyor belt is abnormal, determining the cause of the abnormality of the conveyor belt according to the current image and the current reconstructed image.

[0013] According to another aspect of the embodiments of the present invention, there is also provided a detection device for a conveyor belt, including: a first acquisition unit, configured to acquire a historical image of the conveyor belt, where the historical image is acquired by an image acquisition device; a processing unit, configured to perform compression processing and restoration processing on the historical image to obtain a historical reconstructed image, and construct a reconstruction model according to the historical image and the historical reconstructed image; a second acquisition unit, configured to acquire a current image of the conveyor belt, input the current image into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image; a detection unit, configured to compare the similarity between the current reconstructed image and the current image, and determine whether the conveyor belt is abnormal according to the similarity

[0014] According to still another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium, where the computer-readable storage medium includes a stored program, and wherein the program executes any one of the methods.

[0015] According to another aspect of the embodiments of the present invention, an electronic device is further provided, including: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include those for executing any one of the above methods.

[0016] In the embodiments of the present invention, first, a historical image of the conveyor belt is obtained, and then the historical image is subjected to compression processing and restoration processing to obtain a historical reconstructed image. A reconstruction model is constructed based on the historical image and the historical reconstructed image. Then, a current image of the conveyor belt is obtained, and the current image is input into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image. Finally, the similarity between the current reconstructed image and the current image is compared, and whether the conveyor belt is abnormal is determined according to the similarity. In this solution, a detection scheme for the conveyor belt without abnormal samples for training is proposed. Abnormal detection work can be realized by obtaining images of the conveyor belt in normal operation, without the need for abnormal samples to participate in training the model. Since samples of the conveyor belt in a normal state can be obtained and are also achievable, the accuracy of the detection of the conveyor belt can be guaranteed to be relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings forming a part of this application are used to provide a further understanding of this application. The schematic embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0018] Figure 1 shows a schematic flow chart of a method for detecting a conveyor belt according to an embodiment of the present application;

[0019] Figure 2 shows a schematic structural diagram of a reconstruction model;

[0020] Figure 3 shows a schematic flow chart of training a reconstruction model;

[0021] Figure 4 shows a schematic flow chart of testing a reconstruction model;

[0022] Figure 5 shows another schematic flow chart of testing a reconstruction model;

[0023] Figure 6 shows a schematic structural diagram of a device for detecting a conveyor belt according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0024] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The present application will be described in detail below with reference to the drawings and in conjunction with the embodiments.

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts shall fall within the protection scope of the present application.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances for the embodiments of the present application described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or units that are not clearly listed or are inherent to these processes, methods, products, or devices.

[0027] It should be understood that when an element (such as a layer, film, region, or substrate) is described as being "on" another element, the element can be directly on the other element, or there may also be an intermediate element. Moreover, in the specification and claims, when an element is described as "connected" to another element, the element can be "directly connected" to the other element, or "connected" to the other element through a third element.

[0028] Due to the limitations such as complex design in manual feature extraction, the current method of automatically extracting features by deep learning is widely used. Its main process can be summarized as follows: 1. Extract the image information in a video frame by frame in sequence, and input the conveyor belt images in the normal state as normal training samples and the conveyor belt images in the abnormal state as outlier training samples into the deep learning model for training to obtain the relevant mapping relationship, and this mapping relationship can learn the different characterization information of normal samples and abnormal samples. 2. Divide the features learned in 1 into normal category samples and abnormal category samples through a classifier to achieve the detection of outlier samples. However, during the coal collection process, the abnormality of the conveyor belt will cause a large amount of economic losses. Therefore, it is unrealistic to let the conveyor belt be in an abnormal state in order to collect outlier samples for training the model. Therefore, it is currently difficult to collect outlier samples of the conveyor belt in the abnormal state, resulting in a low detection accuracy of the conveyor belt.

[0029] As described in the background art, it is difficult to collect abnormal samples of the conveyor belt in an abnormal state in the prior art, resulting in a low detection accuracy of the conveyor belt. To solve the above problems, in a typical embodiment of the present application, a detection method, device, computer-readable storage medium and electronic device for a conveyor belt are provided.

[0030] According to an embodiment of the present application, a detection method for a conveyor belt is provided.

[0031] Figure 1 It is a flowchart of the detection method for the conveyor belt according to the embodiment of the present application. As Figure 1 shown, the method includes the following steps:

[0032] Step S101, obtaining historical images of the conveyor belt, where the historical images are collected by an image acquisition device;

[0033] Specifically, historical video information (including historical images) during the normal transmission of the conveyor belt can be collected by an image acquisition device installed above the conveyor belt. The historical video information for a predetermined time period can be collected. The predetermined time period can be 1 hour or other time periods. Samples of historical images are intercepted from the historical video information, and the interception interval can be set according to the actual situation, for example, 1 second.

[0034] Step S102, performing compression processing and restoration processing on the historical images to obtain historical reconstructed images, and constructing a reconstruction model based on the historical images and the historical reconstructed images;

[0035] To further and more efficiently construct a reconstruction model for more efficiently reconstructing an image according to the reconstruction model (reconstruction includes compression processing and restoration processing), in an embodiment of the present application, during the process of constructing the reconstruction model based on the above-mentioned historical image and the above-mentioned historical reconstructed image, the method further includes: constructing an encoder module and a decoder module, where the encoder module is composed of a first number of convolutional neural network layers, the decoder module is composed of a second number of convolutional neural network layers, the encoder module is used to compress the above-mentioned historical image of the first size into the above-mentioned historical image of the second size, and the decoder module is used to restore the above-mentioned historical image of the second size to the above-mentioned historical reconstructed image of the first size, where the above-mentioned first size is larger than the above-mentioned second size; obtaining first mapping information of the encoder module and second mapping information of the decoder module, where the first mapping information refers to the mapping relationship between the above-mentioned historical image of the first size and the above-mentioned historical image of the second size, and the second mapping information refers to the mapping relationship between the above-mentioned historical image of the second size and the above-mentioned historical reconstructed image of the first size; determining a loss function of the above-mentioned reconstruction model according to the above-mentioned historical image, the above-mentioned first mapping information, and the above-mentioned second mapping information.

[0036] In one embodiment, the first number and the second number can be the same number.

[0037] Optionally, sparse constraints can also be imposed on the neurons of the neural network model. Sparse constraints can be imposed on the historical image of the first size, or on the historical image of the second size, or on the historical reconstructed image of the first size. Traditional autoencoders expect to maximize the reconstruction loss of abnormal samples to distinguish normal and abnormal samples. However, in the conveyor belt anomaly detection task involved in this patent, there will be a problem that the reconstruction loss of some abnormal samples is very small in the traditional autoencoder structure, that is, the abnormal image samples are reconstructed into images with a distribution similar to that of normal sample data structures. Based on this, this patent introduces the idea of sparse coding and attempts to overcome this problem through sparse constraints.

[0038] Specifically, the structure of the reconstruction model is as Figure 2 shown, where the encoder module can be a convolutional neural network composed of 3 convolutional neural network layers, and the RGB image samples of 3 channels are mapped from the historical image to the feature encoding through convolution mapping. Correspondingly, the decoder module is similar to the encoder module and is also composed of 3 convolutional neural network layers. It can map the feature encoding to the historical reconstructed image of 3 channels, update the mapping information (including the first mapping information and the second mapping information) according to the encoder module and the decoder module of the reconstruction model, and can also determine the loss function according to the historical image and the historical reconstructed image.

[0039] Activation functions can be used for optimization in the encoder module and the decoder module. Since the swish function can only play a role in deep neural networks, the Hardswish activation function can be used as the activation function of the autoencoder. The Hardswish activation function has advantages such as good numerical stability and fast calculation speed. The expression of the Hardswish activation function is as follows:

[0040]

[0041] Traditional convolutional neural networks generally use functions such as Sigmoid, Tanh, and ReLU as activation functions. The main purpose is to introduce non-linear characteristics into the neural network, that is, to transform a linear regression model into a non-linear model for application to solve the non-linear problems with many hidden layers between the layers of the neural network. Among many activation functions, the swish function can be regarded as a smooth function between a linear function and the ReLU function, and its model effect is better than that of ReLU. However, although the swish non-linear activation function improves the detection accuracy and is more suitable at the academic level, it is not suitable at the application level because the swish function has a higher computational cost, more complex derivative calculation, slower calculation during quantization, and is difficult to be deployed in the conveyor belt anomaly detection task.

[0042] In contrast, the Hardswish non-linear activation function has no obvious difference in accuracy, but has great advantages in practical applications. First, in the quantization mode, the Hardswish function eliminates the potential numerical precision loss caused by different implementations of the approximate Sigmoid shape. Second, in practice, the number of memory accesses is reduced through a piecewise function, which can further reduce the computational cost of the model.

[0043] In order to narrow the distribution distance between the historical reconstructed image of the normal sample and the original image sample (historical image), the gradient descent method can be used to achieve this. First, design the loss function of the reconstruction model, and the formula is:

[0044]

[0045] Among them, Loss represents the loss function, x represents the input historical image, F(.) represents the first mapping information, G(.) represents the second mapping information, λ represents the weight controlling sparsity, j represents the j-th neuron in the feature encoding, c represents the dimension of the feature encoding, ρ represents the sparsity parameter, represents the average activation degree of the feature encoding neurons, is ρ and Relative entropy; in this patent, the value of ρ is set to 0.07 and the value of λ is set to 0.3. Of course, it can also be set to other values according to the actual situation. ρ is a hyperparameter set manually and can be a constant close to 0. Relative entropy is also known as KL divergence and can also be expressed as

[0046] Specifically, the method of autoencoder can be introduced. During the training stage of the reconstruction model, the manifold distribution of normal samples in a latent space can be learned unsupervised. As Figure 3 shown, that is, the training samples (historical images) are compressed by the encoder module of the autoencoder, and the high-dimensional training samples are compressed into a low-dimensional form to obtain feature codes. Then, the compressed data is restored by the decoder module of the autoencoder to obtain the historical reconstructed images. Subsequently, the mapping information of the autoencoder (including the first mapping information and the second mapping information) can be iteratively updated according to the loss function to make the historical images and the historical reconstructed images close. During the test stage of the reconstruction model, as Figure 4 shown, the test samples (historical images or current images) obtain the reconstructed images of the test samples (historical reconstructed images or current reconstructed images) through the mapping information obtained in the training stage. Then, the distribution difference (similarity) between the test samples and the reconstructed images is compared to obtain the output result. If it exceeds the set threshold, the test sample can be determined as an abnormal sample; if it does not exceed the set threshold, the test sample can be determined as a normal sample.

[0047] To further train the reconstruction model to ensure a high accuracy of the obtained reconstruction model and perform reconstruction more efficiently and accurately according to the reconstruction model in the subsequent process, in another embodiment of the present application, after determining the loss function of the reconstruction model according to the above historical images, the above first mapping information, and the above second mapping information, the method further includes: obtaining a plurality of first loss functions, where the first loss function is the loss function corresponding to the first mapping information, and the plurality of first loss functions are obtained by inputting a third quantity of the above historical images as a training set into the reconstruction model; obtaining the average value of a plurality of second loss functions, where the second loss function is the loss function corresponding to the second mapping information, and the plurality of second loss functions are obtained by inputting a fourth quantity of the above historical images as a validation set into the reconstruction model; determining the first loss function with the smallest difference from the above average value as the target loss function, and determining the first mapping information corresponding to the target loss function as the target mapping information.

[0048] Specifically, the collected historical image dataset can be divided into a training set and a validation set according to a predetermined ratio. The predetermined ratio can be set according to the actual situation. For example, in a ratio of 8:2. In one embodiment, a total of 3,600 historical images are collected. Then, 2,880 historical images are used as the training set, and 720 historical images are used as the validation set. The collected historical images are preprocessed. Since the coal mine conveyor belt mainly involves brightness changes and does not involve factors such as translation and rotation, the preprocessing can at least include brightness change processing.

[0049] Specifically, the target mapping information can be determined through iterative processing. The following is a specific description of the process:

[0050] In the first round of iteration, first initialize the first mapping information and the second mapping information of the reconstruction model, including: randomly select a historical image from the training set and input it into the reconstruction model. Obtain the historical reconstructed image through the initialized first mapping information and the second mapping information. Calculate the gradient direction using the loss function and update the first mapping information and the second mapping information through the set learning rate. The learning rate can be set according to the actual situation. For example, it is 0.001. Take the updated two mapping information as the initialized first mapping information and the second mapping information of the reconstruction model. Input another historical image from the training set into the reconstruction model and repeat the above updated steps until the first mapping information and the second mapping information corresponding to all the training samples participating in the training in the training set are obtained, and store the corresponding first mapping information and the second mapping information;

[0051] Input all the historical images on the validation set into the reconstruction model, and obtain the historical reconstructed image corresponding to the historical image through the mapping information obtained by training. Calculate the distribution difference between all the historical images on the validation set and the corresponding historical reconstructed images, obtain the corresponding second loss function, and calculate the average value. The formula is as follows:

[0052]

[0053] where L represents the average value, n represents the number of historical images in the validation set, and x i represents the i-th historical image in the validation set;

[0054] Iterate again, repeat the above initialization steps and validation steps. In the second round of iteration, the initialized first mapping information and the initialized second mapping information are the first mapping information and the second mapping information saved in the first round of iteration. Subsequently, the iteration can be performed multiple times to ensure that the accuracy of the first mapping information and the second mapping information is relatively high. The number of iterations can be set according to the actual situation. For example, 500 times.

[0055] Finally, the first loss function corresponding to the j-th round with the smallest difference in the average value is selected as the target loss function, and the first mapping information corresponding to the target loss function is the target mapping information. Among them, when the number of iterations is 500, L j = min(L1, L2,...L k ), where k represents the number of iterations.

[0056] Step S103: Obtain the current image of the conveyor belt, input the current image into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image;

[0057] Step S104: Compare the similarity between the current reconstructed image and the current image, and determine whether the conveyor belt is abnormal according to the similarity.

[0058] Specifically, after the reconstruction model is constructed, the reconstruction model can also be tested. As Figure 5 shown, the testing stage mainly realizes the detection of abnormal samples by constructing a classifier for normal samples (normal historical images) and abnormal samples (abnormal historical images) through the trained target mapping information. The classifier can be realized by setting a judgment threshold (i.e., similarity threshold) for abnormal samples.

[0059] For the parameters trained in the training stage of the reconstruction model, ideally, the distribution difference between the historical images working normally and the historical reconstructed images output by the reconstruction model is small (the similarity is large) or even 0, and the distribution difference between the historical images working abnormally and the historical reconstructed images output by the reconstruction model is large (the similarity is small). Therefore, the conveyor belt can be detected for abnormalities by setting corresponding thresholds. If the similarity threshold is exceeded, the historical image is determined to be an abnormal sample; if the similarity threshold is not exceeded, the historical image is determined to be a normal sample.

[0060] Specifically, the threshold adopted by the classifier can be calculated by the following formula: Ω = max(L1, L2,..., L k ), where L1, L2, and L k are all the loss values of the images in the validation set after being reconstructed by the autoencoder and the original images. η is a set parameter, which can be 0.5 or any other feasible value.

[0061] Since the field of view captured by the image acquisition device in reality includes the second region, the distribution difference of abnormal samples may be caused by non-conveyor belt abnormalities, such as a bird flying through the second region, etc. In this case, exceeding the similarity threshold may not require triggering an alarm. Therefore, it is possible to further efficiently and accurately determine whether the conveyor belt is abnormal based on the position of the target region. In a specific embodiment of the present application, determining whether the conveyor belt is abnormal according to the above similarity includes: when the above similarity is greater than or equal to the similarity threshold, determining that the above conveyor belt is normal; when the above similarity is less than the above similarity threshold, obtaining the first region and the second region in the above current image, where the first region refers to the region where the above conveyor belt is located, and the second region refers to the region other than the region where the above conveyor belt is located; determining whether the conveyor belt is abnormal according to the position of the target region, where the target region refers to the region in the above current image and the above current reconstructed image where the above similarity is less than the above similarity threshold.

[0062] To further efficiently and accurately determine the first region and the second region in the current image to further efficiently and accurately determine whether the conveyor belt is abnormal, in another specific embodiment of the present application, obtaining the first region and the second region in the above current image includes: obtaining a plurality of predetermined coordinate points, where the predetermined coordinate points are the position coordinate points in the above current image for marking the above first region; determining the region enclosed by the plurality of above predetermined coordinate points as the above first region; determining the region in the above current image other than the above first region as the above second region.

[0063] Specifically, the position coordinate points of the first region can be manually marked first. The first region is displayed by the image acquisition device with a plurality of predetermined coordinate points, such as the upper left coordinate point, the upper right coordinate point, the lower left coordinate point, and the lower right coordinate point.

[0064] Subtract the gray value of the pixel point corresponding to the historical image of the abnormal sample from the gray value of the pixel point corresponding to the previous frame of the historical image, and then take the absolute value to obtain a difference image. The formula can be D n (x,y)=f n (x,y)-f n-1 (x,y), where D n (x,y) represents the pixel point corresponding to the difference image, f n (x,y) represents the pixel point corresponding to the current image, f n-1 (x,y) represents the pixel point corresponding to the previous frame of the historical image of the current image.

[0065] Perform binarization on the absolute value D n (x,y) of the difference value to obtain where T is a threshold hyperparameter, which can be 25.

[0066] The target mapping information of the set model and the reconstruction model of the classifier can be run on the server. The video information of the conveyor belt collected by the image acquisition device is uploaded to the server. The server takes the obtained video information as current images in the form of 20 frames and uses them as input images to the reconstruction model and the classifier. When the classifier identifies an abnormal sample, it further checks for abnormalities through the inter-frame difference method. If it is determined to be abnormal, the abnormal input image is stored on the server. At the same time, the PLC variable is written through the OPC interface, and the conveyor belt is paused and an alarm signal is sent through the set PLC control program, and the warning function is realized through the corresponding alarm device. On the other hand, the stored abnormal images and alarm information are uploaded to the intelligent integrated management and control platform to assist relevant personnel in judging abnormal information.

[0067] In the case of initially determining the abnormality of the conveyor belt according to the similarity threshold, it is also possible to further efficiently and accurately determine whether the conveyor belt is abnormal according to the position of the target area of the conveyor belt. In another specific embodiment of the present application, determining whether the above conveyor belt is abnormal according to the position of the target area includes: determining that the conveyor belt is abnormal when the target area is within the first area; determining that the conveyor belt is normal when the target area is within the second area; and determining that the conveyor belt is abnormal when part of the target area is within the first area and part is within the second area.

[0068] In the case where it has been determined that the conveyor belt is abnormal, it is also possible to determine the cause of the abnormality of the conveyor belt, so that the fault location of the conveyor belt can be determined in time, and the staff can be prompted in time about the reason for the abnormality of the conveyor belt, which is convenient for the staff to repair. In another embodiment of the present application, after determining whether the conveyor belt is abnormal according to the above similarity, the method further includes: in the case of determining that the conveyor belt is abnormal, determining the cause of the abnormality of the conveyor belt according to the current image and the current reconstructed image.

[0069] Specifically, in the case of determining that the conveyor belt is abnormal, it is possible to first determine whether the conveyor belt is running off track according to the edge detection algorithm. In the case of running off track, it is determined that the cause of the abnormality is that the conveyor belt is running off track. In the case where the conveyor belt is not running off track, it is possible to determine whether there is foreign matter on the conveyor belt according to the image recognition algorithm. In the case of having foreign matter, it is determined that the cause of the abnormality is that there is foreign matter on the conveyor belt.

[0070] It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0071] In the above method, first, the historical image of the conveyor belt is obtained. Then, the historical image is subjected to compression processing and restoration processing to obtain a historical reconstructed image. A reconstruction model is constructed based on the historical image and the historical reconstructed image. Next, the current image of the conveyor belt is obtained and input into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image. Finally, the similarity between the current reconstructed image and the current image is compared, and whether the conveyor belt is abnormal is determined according to the similarity. In this solution, a detection scheme for the conveyor belt without abnormal samples for training is proposed. Abnormal detection work can be realized by obtaining the images of the conveyor belt working normally, without the need for abnormal samples to participate in training the model. Since the samples of the conveyor belt in the normal state can be obtained and realized, the accuracy of the detection of the conveyor belt can be guaranteed to be relatively high.

[0072] The embodiment of the present application further provides a detection device for a conveyor belt. It should be noted that the detection device for the conveyor belt in the embodiment of the present application can be used to execute the detection method for the conveyor belt provided in the embodiment of the present application. The following introduces the detection device for the conveyor belt provided in the embodiment of the present application.

[0073] Figure 6 is a schematic diagram of the detection device for the conveyor belt according to the embodiment of the present application. As Figure 6 shown, the device includes:

[0074] A first acquisition unit 10, configured to acquire the historical image of the conveyor belt, where the historical image is acquired by an image acquisition device;

[0075] A processing unit 20, configured to perform compression processing and restoration processing on the historical image to obtain a historical reconstructed image, and construct a reconstruction model according to the historical image and the historical reconstructed image;

[0076] To further and more efficiently construct a reconstruction model for more efficiently reconstructing an image according to the reconstruction model (reconstruction includes compression processing and restoration processing), in an embodiment of the present application, the above device further includes a construction unit, a third acquisition unit, and a first determination unit. The construction unit is configured to construct an encoder module and a decoder module during the process of constructing the reconstruction model according to the above historical image and the above historical reconstructed image. Among them, the above encoder module is composed of a first number of convolutional neural network layers, and the above decoder module is composed of a second number of convolutional neural network layers. The above encoder module is configured to compress the above historical image of the first size into the above historical image of the second size, and the above decoder module is configured to restore the above historical image of the second size to the above historical reconstructed image of the first size, where the above first size is greater than the above second size; the third acquisition unit is configured to acquire first mapping information of the above encoder module and acquire second mapping information of the above decoder module. The above first mapping information refers to the mapping relationship between the above historical image of the first size and the above historical image of the second size, and the above second mapping information refers to the mapping relationship between the above historical image of the second size and the above historical reconstructed image of the first size; the first determination unit is configured to determine the loss function of the above reconstruction model according to the above historical image, the above first mapping information, and the above second mapping information.

[0077] To further train the reconstruction model to ensure that the obtained reconstruction model has a high accuracy and can perform reconstruction more efficiently and accurately according to the reconstruction model in the future, in another embodiment of the present application, the above device further includes a fourth acquisition unit, a fifth acquisition unit, and a second determination unit. The fourth acquisition unit is configured to acquire a plurality of first loss functions after determining the loss function of the above reconstruction model according to the above historical image, the above first mapping information, and the above second mapping information. The above first loss function is the loss function corresponding to the above first mapping information, and the plurality of above first loss functions are obtained by inputting a third number of the above historical images as a training set into the above reconstruction model; the fifth acquisition unit is configured to acquire the average value of a plurality of second loss functions. The above second loss function is the loss function corresponding to the above second mapping information, and the plurality of above second loss functions are obtained by inputting a fourth number of the above historical images as a validation set into the above reconstruction model; the second determination unit is configured to determine the first loss function with the smallest difference from the above average value as the target loss function and determine the above first mapping information corresponding to the target loss function as the target mapping information.

[0078] The second acquisition unit 30 is configured to acquire the current image of the above conveyor belt, input the above current image into the above reconstruction model for compression processing and restoration processing, and obtain the current reconstructed image;

[0079] The detection unit 40 is configured to compare the similarity between the current reconstructed image and the current image, and determine whether the conveyor belt is abnormal according to the similarity.

[0080] Since the field of view captured by the image acquisition device in reality includes the second area, the distribution difference of abnormal samples may be caused by non-conveyor belt abnormalities, such as a bird flying through the second area, etc. In this case, exceeding the similarity threshold may not need to trigger an alarm. Therefore, it is possible to further efficiently and accurately determine whether the conveyor belt is abnormal according to the position of the target area. In a specific embodiment of the present application, the detection unit includes a first determination module, an acquisition module, and a second determination module. The first determination module is configured to determine that the conveyor belt is normal when the similarity is greater than or equal to the similarity threshold; the acquisition module is configured to, when the similarity is less than the similarity threshold, acquire a first area and a second area in the current image, where the first area refers to the area where the conveyor belt is located, and the second area refers to the area other than the area where the conveyor belt is located; the second determination module is configured to determine whether the conveyor belt is abnormal according to the position of the target area, where the target area refers to the area where the similarity between the current image and the current reconstructed image is less than the similarity threshold.

[0081] To further efficiently and accurately determine the first area and the second area in the current image, so as to further efficiently and accurately determine whether the conveyor belt is abnormal, in another specific embodiment of the present application, the acquisition module includes an acquisition sub-module, a first determination sub-module, and a second determination sub-module. The acquisition sub-module is configured to acquire a plurality of predetermined coordinate points, where the predetermined coordinate points are the position coordinate points in the current image for marking the first area; the first determination sub-module is configured to determine the area enclosed by the plurality of predetermined coordinate points as the first area; the second determination sub-module is configured to determine the area in the current image other than the first area as the second area.

[0082] When initially determining that the conveyor belt is abnormal according to the similarity threshold, it is also possible to further efficiently and accurately determine whether the conveyor belt is abnormal according to the position of the target area of the conveyor belt. In another specific embodiment of the present application, the second determination module includes a third determination sub-module, a fourth determination sub-module, and a fifth determination sub-module. The third determination sub-module is configured to determine that the conveyor belt is abnormal when the target area is within the first area; the fourth determination sub-module is configured to determine that the conveyor belt is normal when the target area is within the second area; the fifth determination sub-module is configured to determine that the conveyor belt is abnormal when part of the target area is within the first area and part is within the second area.

[0083] In the case where the abnormality of the conveyor belt has been determined, the cause of the abnormality of the conveyor belt can also be determined, so that the fault location of the conveyor belt can be determined in a timely manner, and the staff can be prompted in a timely manner about the reason for the abnormality of the conveyor belt, so as to facilitate the maintenance by the staff. In another embodiment of the present application, the above device further includes a third determination unit, and the third determination unit is used to determine the cause of the abnormality of the conveyor belt according to the current image and the current reconstructed image after determining whether the conveyor belt is abnormal according to the above similarity.

[0084] In the above device, the first acquisition unit is used to acquire the historical image of the conveyor belt, the processing unit is used to perform compression processing and restoration processing on the historical image to obtain the historical reconstructed image, and construct a reconstruction model according to the historical image and the historical reconstructed image. The second acquisition unit is used to acquire the current image of the conveyor belt, input the current image into the reconstruction model for compression processing and restoration processing to obtain the current reconstructed image. The detection unit is used to compare the similarity between the current reconstructed image and the current image, and determine whether the conveyor belt is abnormal according to the similarity. In this solution, a detection solution for a conveyor belt trained without abnormal samples is proposed. Abnormality detection work can be realized by acquiring the images of the conveyor belt working normally, and abnormal samples do not need to participate in the training model. Since the samples of the conveyor belt in the normal state can be acquired and realized, the accuracy of the detection of the conveyor belt can be guaranteed to be relatively high.

[0085] The above detection device for the conveyor belt includes a processor and a memory. The above first acquisition unit, processing unit, second acquisition unit, detection unit, etc. are all stored in the memory as program units, and the processor executes the above program units stored in the memory to realize the corresponding functions.

[0086] The processor contains a kernel, and the kernel retrieves the corresponding program units from the memory. One or more kernels can be set, and the accuracy of conveyor belt detection can be improved by adjusting the kernel parameters.

[0087] The memory may include non-permanent memory in a computer-readable medium, forms such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0088] An embodiment of the present invention provides a computer-readable storage medium, on which a program is stored, and when the program is executed by a processor, the above detection method for the conveyor belt is realized.

[0089] An embodiment of the present invention provides a processor, and the above processor is used to run a program, wherein when the above program runs, the above detection method for the conveyor belt is executed.

[0090] The present application also provides an electronic device, including one or more processors, a memory, and one or more programs. Among them, the above one or more programs are stored in the above memory and are configured to be executed by the above one or more processors. The above one or more programs include those for executing any of the above methods.

[0091] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, at least the following steps are implemented:

[0092] Step S101: Obtain a historical image of the conveyor belt, where the historical image is collected by an image acquisition device;

[0093] Step S102: Perform compression processing and restoration processing on the above historical image to obtain a historical reconstructed image, and construct a reconstruction model based on the above historical image and the above historical reconstructed image;

[0094] Step S103: Obtain the current image of the above conveyor belt, input the current image into the above reconstruction model for compression processing and restoration processing, and obtain a current reconstructed image;

[0095] Step S104: Compare the similarity between the above current reconstructed image and the above current image, and determine whether the conveyor belt is abnormal according to the above similarity.

[0096] The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0097] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with at least the following method steps:

[0098] Step S101: Obtain a historical image of the conveyor belt, where the historical image is collected by an image acquisition device;

[0099] Step S102: Perform compression processing and restoration processing on the above historical image to obtain a historical reconstructed image, and construct a reconstruction model based on the above historical image and the above historical reconstructed image;

[0100] Step S103: Obtain the current image of the above conveyor belt, input the current image into the above reconstruction model for compression processing and restoration processing, and obtain a current reconstructed image;

[0101] Step S104: Compare the similarity between the above current reconstructed image and the above current image, and determine whether the conveyor belt is abnormal according to the above similarity.

[0102] In the above embodiments of the present invention, the descriptions of the respective embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0103] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the above division of units can be a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0104] The units described above as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0105] In addition, the functional units in the various embodiments of the present invention can be integrated into a processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0106] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the above methods in the various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs and other various media that can store program codes.

[0107] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0108] 1) The conveyor belt detection method of this application first obtains the historical image of the conveyor belt, then performs compression processing and restoration processing on the historical image to obtain a historical reconstructed image, constructs a reconstruction model based on the historical image and the historical reconstructed image, then obtains the current image of the conveyor belt, inputs the current image into the reconstruction model for compression processing and restoration processing to obtain the current reconstructed image, and finally compares the similarity between the current reconstructed image and the current image, and determines whether the conveyor belt is abnormal according to the similarity. In this solution, a conveyor belt detection solution without abnormal samples for training is proposed. Abnormal detection work can be achieved by obtaining the images of the conveyor belt in normal operation, without the need for abnormal samples to participate in training the model. Since the samples of the conveyor belt in the normal state can be obtained and are also achievable, the accuracy of the conveyor belt detection can be guaranteed to be relatively high.

[0109] 2) The conveyor belt detection device of this application, the first acquisition unit is used to obtain the historical image of the conveyor belt, the processing unit is used to perform compression processing and restoration processing on the historical image to obtain a historical reconstructed image, constructs a reconstruction model based on the historical image and the historical reconstructed image, the second acquisition unit is used to obtain the current image of the conveyor belt, inputs the current image into the reconstruction model for compression processing and restoration processing to obtain the current reconstructed image, and the detection unit is used to compare the similarity between the current reconstructed image and the current image, and determines whether the conveyor belt is abnormal according to the similarity. In this solution, a conveyor belt detection solution without abnormal samples for training is proposed. Abnormal detection work can be achieved by obtaining the images of the conveyor belt in normal operation, without the need for abnormal samples to participate in training the model. Since the samples of the conveyor belt in the normal state can be obtained and are also achievable, the accuracy of the conveyor belt detection can be guaranteed to be relatively high.

[0110] The above are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A detection method for a conveyor belt, characterized in that, Including: Obtain historical images of the conveyor belt, where the historical images are collected by an image acquisition device; Perform compression processing and restoration processing on the historical images to obtain historical reconstructed images, and construct a reconstruction model based on the historical images and the historical reconstructed images; Obtain the current image of the conveyor belt, input the current image into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image; Compare the similarity between the current reconstructed image and the current image, and determine whether the conveyor belt is abnormal according to the similarity; In the process of constructing the reconstruction model based on the historical images and the historical reconstructed images, the method further includes: constructing an encoder module and a decoder module, where the encoder module is composed of a first number of convolutional neural network layers, and the decoder module is composed of a second number of convolutional neural network layers. The encoder module is used to compress the historical image of the first size into the historical image of the second size, and the decoder module is used to restore the historical image of the second size to the historical reconstructed image of the first size, where the first size is larger than the second size; obtain the first mapping information of the encoder module, and obtain the second mapping information of the decoder module. The first mapping information refers to the mapping relationship between the historical image of the first size and the historical image of the second size, and the second mapping information refers to the mapping relationship between the historical image of the second size and the historical reconstructed image of the first size; determine the loss function of the reconstruction model according to the historical image, the first mapping information, and the second mapping information; After determining the loss function of the reconstruction model according to the historical image, the first mapping information, and the second mapping information, the method further includes: obtaining a plurality of first loss functions, where the first loss function is the loss function corresponding to the first mapping information, and the plurality of first loss functions are obtained by inputting a third number of the historical images as a training set into the reconstruction model; obtaining the average value of a plurality of second loss functions, where the second loss function is the loss function corresponding to the second mapping information, and the plurality of second loss functions are obtained by inputting a fourth number of the historical images as a validation set into the reconstruction model; determining the first loss function with the smallest difference from the average value as the target loss function, and determining the first mapping information corresponding to the target loss function as the target mapping information; 2. The method according to claim 1, wherein, Determining whether the conveyor belt is abnormal according to the similarity includes: When the similarity is greater than or equal to the similarity threshold, determine that the conveyor belt is normal; When the similarity is less than the similarity threshold, obtain a first region and a second region in the current image, where the first region refers to the region where the conveyor belt is located, and the second region refers to the region where the conveyor belt is not located; Determine whether the conveyor belt is abnormal according to the position of the target area, where the target area refers to the area where the similarity between the current image and the current reconstructed image is less than the similarity threshold.

3. The method according to claim 2, characterized in that, Obtain a first area and a second area in the current image, including: Obtain a plurality of predetermined coordinate points, where the predetermined coordinate points are the position coordinate points used to mark the first area in the current image; Determine the area enclosed by the plurality of predetermined coordinate points as the first area; Determine the area in the current image other than the first area as the second area.

4. The method according to claim 2, characterized in that, Determine whether the conveyor belt is abnormal according to the position of the target area, including: When the target area is within the first area, determine that the conveyor belt is abnormal; When the target area is within the second area, determine that the conveyor belt is normal; When part of the target area is within the first area and part is within the second area, determine that the conveyor belt is abnormal.

5. The method according to claim 1, wherein After determining whether the conveyor belt is abnormal according to the similarity, the method further includes: When it is determined that the conveyor belt is abnormal, determine the cause of the abnormality of the conveyor belt according to the current image and the current reconstructed image.

6. A detection device for a conveyor belt, characterized in that, Including: A first acquisition unit for acquiring a historical image of the conveyor belt, where the historical image is acquired by an image acquisition device; A processing unit for performing compression processing and restoration processing on the historical image to obtain a historical reconstructed image, and constructing a reconstruction model according to the historical image and the historical reconstructed image; A second acquisition unit for acquiring the current image of the conveyor belt, inputting the current image into the reconstruction model for compression processing and restoration processing to obtain a current reconstructed image; A detection unit for comparing the similarity between the current reconstructed image and the current image, and determining whether the conveyor belt is abnormal according to the similarity. The device further includes a construction unit, a third acquisition unit, and a first determination unit. The construction unit is configured to construct an encoder module and a decoder module during the process of constructing a reconstruction model based on the historical image and the historical reconstructed image. Among them, the encoder module is composed of a first number of convolutional neural network layers, and the decoder module is composed of a second number of convolutional neural network layers. The encoder module is used to compress the historical image of the first size into the historical image of the second size, and the decoder module is used to restore the historical image of the second size to the historical reconstructed image of the first size, where the first size is larger than the second size. The third acquisition unit is configured to acquire first mapping information of the encoder module and second mapping information of the decoder module. The first mapping information refers to the mapping relationship between the historical image of the first size and the historical image of the second size, and the second mapping information refers to the mapping relationship between the historical image of the second size and the historical reconstructed image of the first size. The first determination unit is configured to determine the loss function of the reconstruction model according to the historical image, the first mapping information, and the second mapping information. The device further includes a fourth acquisition unit, a fifth acquisition unit, and a second determination unit. The fourth acquisition unit is configured to acquire a plurality of first loss functions after determining the loss function of the reconstruction model according to the historical image, the first mapping information, and the second mapping information. The first loss function is the loss function corresponding to the first mapping information, and the plurality of first loss functions are obtained by inputting a third number of the historical images as a training set into the reconstruction model. The fifth acquisition unit is configured to acquire the average value of a plurality of second loss functions. The second loss function is the loss function corresponding to the second mapping information, and the plurality of second loss functions are obtained by inputting a fourth number of the historical images as a validation set into the reconstruction model. The second determination unit is configured to determine the first loss function with the smallest difference from the average value as the target loss function, and determine the first mapping information corresponding to the target loss function as the target mapping information.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, where the program executes the method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, Comprising: One or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors. The one or more programs include those for executing the method according to any one of claims 1 to 5.

Images