Detection Method and Device for Belt Deviation of Mine Roadway, Processor and Electronic Device

Abstract

The present application provides a method and device for detecting the deviation of a mine conveyor belt, a processor, and an electronic device. By acquiring an initial mine image, performing multi-feature enhancement processing on the initial mine image, and obtaining a feature fusion image; controlling the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network; based on a contour repair algorithm, repairing the contour of the mine conveyor belt in the semantic segmentation image to determine the complete contour corresponding to the mine conveyor belt; determining the warning line in the semantic segmentation image, and based on the complete contour and the warning line, determining whether there is a deviation phenomenon of the mine conveyor belt, and triggering an alarm when there is a deviation phenomenon of the mine conveyor belt, which solves the technical problems that the traditional image segmentation method in the prior art has high efficiency but low accuracy, and the semantic segmentation method of deep learning has high accuracy but poor efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing, and more particularly, to a method and device for detecting the deviation of a mine conveyor belt, a processor, and an electronic device. Background Art

[0002] With the progress of artificial intelligence, the research on computer vision (CV) has received extensive attention from scholars at home and abroad. The challenge in the field of computer vision lies in the semantic gap. Humans can relatively easily obtain the semantic information contained in pictures, but computers can only understand images as numbers between 0 and 255. The field of computer vision aims to enable computers to have the ability to perceive the surrounding environment using vision like humans.

[0003] Computer vision research mainly has the following four types of tasks: The first type of task is image classification, and the image classification task can calculate the number of categories contained in the image. The second type of task is image object detection. For a given image, object detection identifies the object and finds its precise location. The third type of task is image semantic segmentation. Semantic segmentation classifies the physical objects contained in the image by category, labels the category information for all pixel points of the image, and the pixels of the same physical object should use the same category label. The fourth type of task is image instance segmentation. Instance segmentation not only requires labeling the pixel categories but also requires further distinguishing physical objects belonging to the same major category.

[0004] According to the development and evolution process of semantic segmentation technology, these works can be roughly divided into traditional semantic segmentation algorithms and deep learning-based semantic segmentation algorithms. Traditional semantic segmentation algorithms mainly divide images into different regions according to features such as the red-green-blue (RGB) color, texture, grayscale, and geometric shape of the image. There is also a semantic segmentation algorithm based on random forests, which explores the correlation of semantic context using local feature histograms of the image for modeling and constructs the semantic segmentation result by using random forests for classification weighting. In addition, by combining the contour detection method with a separator, the modified conjugate gradient (MCG) algorithm is proposed. The image is segmented into multiple blocks using the contour detection algorithm and then further segmented using a random classifier to achieve image semantic segmentation.

[0005] The results of traditional segmentation methods are relatively rough with low accuracy, but they are highly efficient and relatively simple to implement. In contrast, deep learning-based methods mainly use self-learning methods to extract image features and achieve pixel classification of images through continuous training. For example, the SegNet network based on the encoding-decoding method obtains a sparse feature map through image convolution encoding and then restores it to a dense segmentation map through deconvolution. There is also a generative adversarial image segmentation network model based on multi-level functions and multi-scales. By using a discriminator to deeply learn the local attributes and global structure in the image segmentation process, the relative relationship between pixels can be obtained to achieve pixel classification. The segmentation accuracy of deep learning-based semantic segmentation algorithms is basically proportional to the model complexity. Therefore, although its accuracy is relatively high compared to traditional methods, its model efficiency and interpretability are poor, which also limits to a certain extent the implementation of deep learning-based semantic segmentation methods. Summary of the Invention

[0006] The main objective of this application is to provide a method and device for detecting the deviation of a mine conveyor belt, a processor, and an electronic device, so as to at least solve the technical problem in the prior art that traditional image segmentation methods are highly efficient but have low accuracy, while deep learning-based semantic segmentation methods have high accuracy but poor efficiency and interpretability.

[0007] To achieve the above objective, according to one aspect of this application, a method for detecting the deviation of a mine conveyor belt is provided, which specifically includes: obtaining an initial mine image, performing multi-feature enhancement processing on the initial mine image, and obtaining a feature fusion image; controlling the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine conveyor belt; repairing the contour of the mine conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine conveyor belt; determining the warning line in the semantic segmentation image, and based on the complete contour and the warning line, determining whether the mine conveyor belt has a deviation phenomenon, and triggering an alarm when the mine conveyor belt has a deviation phenomenon, where the warning line is two lines composed of continuous pixel points on the left and right.

[0008] Further, performing multi-feature enhancement processing on the initial mine image and obtaining a feature fusion image includes: determining multiple image features, where the multiple image features at least include spatial color, edge information, and contrast, and the spatial color at least includes the RGB original image, image hue, image saturation, and image brightness; controlling the initial mine image to perform feature enhancement processing in multiple dimensions corresponding to the multiple image features to obtain multiple groups of data corresponding to the multiple dimensions; and fusing the multiple groups of data to obtain a feature fusion image.

[0009] Further, before the control feature fusion image is input into the target wavelet neural network to obtain the semantic segmentation image output by the target wavelet neural network, the method includes: obtaining an initial wavelet neural network, where the initial wavelet neural network is composed of an input layer, a hidden layer, and an output layer, the initial wavelet neural network contains multiple wavelet elements, and the structure in the initial wavelet neural network is a compact structure, and the compact structure integrates wavelet functions into the neural network; by using the gradient descent method, correcting the network weights and the parameters of the wavelet functions in the initial wavelet neural network, and obtaining the corrected wavelet neural network; determining the corrected wavelet neural network as the target wavelet neural network.

[0010] Further, based on the contour repair algorithm, repair the contour of the mine conveyor belt in the semantic segmentation image to determine the complete contour corresponding to the mine conveyor belt, including: determining the contour pixel points in the semantic segmentation image through the contour repair algorithm, and repairing the contour of the mine conveyor belt based on the contour pixel points to obtain the complete contour, where the contour pixel points are the pixel points corresponding to the contour of the mine conveyor belt.

[0011] Further, determining the contour pixel points in the semantic segmentation image through the contour repair algorithm, and repairing the contour of the mine conveyor belt based on the contour pixel points, including: sequentially traversing each pixel point in the semantic segmentation image, and respectively determining whether each pixel point is a contour pixel point; if the pixel point is a contour pixel point, labeling the pixel point, and determining all the labeled pixel points in the semantic segmentation image; connecting all the labeled pixel points to obtain a connected curve, and repairing the contour of the mine conveyor belt based on the connected curve.

[0012] Further, determining whether the mine conveyor belt has a deviation phenomenon based on the complete contour and the warning line, including: determining the contour pixel points corresponding to the complete contour and the warning line pixel points corresponding to the warning line; when there is an overlapping phenomenon between the contour pixel points and the warning line pixel points, determining that the mine conveyor belt has a deviation phenomenon.

[0013] Further, obtaining the initial mine image, including: collecting the working video of the mine, and obtaining the initial mine image based on the working video.

[0014] To achieve the above object, according to one aspect of the present application, a detection device for the deviation of a mine tunnel conveyor belt is provided, including: a first acquisition unit, configured to acquire an initial mine tunnel image, perform multi-feature enhancement processing on the initial mine tunnel image, and acquire a feature fusion image; a first control unit, configured to control the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, wherein the semantic segmentation image includes the contour of the mine tunnel conveyor belt; a first determination unit, configured to repair the contour of the mine tunnel conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine tunnel conveyor belt; a second determination unit, configured to determine the warning line in the semantic segmentation image, and based on the complete contour and the warning line, determine whether there is a deviation phenomenon of the mine tunnel conveyor belt, and trigger an alarm when there is a deviation phenomenon of the mine tunnel conveyor belt, wherein the warning line is two left and right lines composed of continuous pixel points.

[0015] According to another aspect of the present application, a processor is provided, and the processor is used to run a program, wherein when the program runs, it executes a detection method for the deviation of a mine tunnel conveyor belt.

[0016] According to another aspect of the present application, an electronic device is provided, including: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a detection method for executing the deviation of a mine tunnel conveyor belt.

[0017] Applying the technical solution of the present application, by obtaining an initial mine roadway image, performing multi-feature enhancement processing on the initial mine roadway image, and obtaining a feature fusion image; controlling the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, wherein the semantic segmentation image includes the contour of the mine roadway conveyor belt; repairing the contour of the mine roadway conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine roadway conveyor belt; determining the warning line in the semantic segmentation image, and determining whether the mine roadway conveyor belt has a deviation phenomenon based on the complete contour and the warning line, and triggering an alarm when the mine roadway conveyor belt has a deviation phenomenon. The warning line is two left and right lines composed of continuous pixel points. Therefore, the present application first considers a variety of traditional image features, and then uses a wavelet neural network, which combines the time-frequency localization characteristics of wavelet transform and the self-learning characteristics of neural network. Compared with traditional RBF and BP networks, it has higher degrees of freedom, elasticity and plasticity, so it has good image approximation and pattern recognition and classification capabilities. In the relatively complex real-time scenario in the mine roadway, it can ensure the effectiveness of subsequent feature extraction. Finally, a contour repair method based on contour information is introduced according to the shape characteristics of the conveyor belt, refining the segmentation result of the target boundary, solving the technical problems in the prior art that the traditional image segmentation method has high efficiency but low accuracy, and the semantic segmentation method of deep learning has high accuracy but poor efficiency and interpretability, and further achieving the technical effect of improving the segmentation accuracy. Brief Description of the Drawings

[0018] The specification drawings forming a part of the present application are used to provide a further understanding of the present application. The schematic embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0019] Figure 1 is a hardware structure block diagram of a mobile terminal for a method of detecting deviation of a mine roadway conveyor belt provided by an embodiment of the present invention;

[0020] Figure 2 is a flowchart of a method of detecting deviation of a mine roadway conveyor belt provided by an embodiment of the present application;

[0021] Figure 3 shows a network topology structure diagram of a wavelet neural network provided by the present application;

[0022] Figure 4 is a schematic diagram of a device for detecting deviation of a mine roadway conveyor belt provided by an embodiment of the present application. Detailed Description of the Embodiments

[0023] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments may be combined with each other. The following will describe the present application in detail with reference to the drawings and in conjunction with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances for the embodiments of the present application described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0026] As introduced in the background art, there are technical problems in the prior art that traditional image segmentation methods are efficient but have low accuracy, while deep learning semantic segmentation methods have high accuracy but poor efficiency and interpretability. To solve the above problems, the embodiments of the present application provide a method and device for detecting the deviation of a mine tunnel conveyor belt, a processor, and an electronic device.

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention.

[0028] The method embodiments provided in the embodiments of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 is a hardware structure block diagram of a mobile terminal for a method of detecting the deviation of a mine tunnel conveyor belt according to an embodiment of the present invention. As Figure 1 shown, the mobile terminal may include one or more ( Figure 1 only one is shown in ) processor 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data. Among them, the above-mentioned mobile terminal may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that,Figure 1 The structure shown is only schematic and does not limit the structure of the above-mentioned mobile terminal. For example, the mobile terminal may further include more or fewer components than those shown in Figure 1 and may have a configuration different from that shown in Figure 1 .

[0029] The memory 104 can be used to store computer programs. For example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implements the above method. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the mobile terminal through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof. The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include the wireless network provided by the communication provider of the mobile terminal. In one instance, the transmission device 106 includes a network adapter (abbreviated as NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a method for detecting the deviation of a mine tunnel conveyor belt running on a mobile terminal, a computer terminal, or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0031] Figure 2 is a flowchart of a method for detecting the deviation of a mine tunnel conveyor belt according to an embodiment of the present application. As Figure 2 shown, the method includes the following steps:

[0032] Step S201, obtain an initial mine tunnel image, perform multi-feature enhancement processing on the initial mine tunnel image, and obtain a feature fusion image.

[0033] Specifically, image data of the mine conveyor belt is collected by a camera sensor installed above the coal mine conveyor belt. In this application, by enhancing the mine image in multiple feature dimensions, a fused image with multiple features is obtained, thereby improving the ability of the subsequent semantic segmentation network to acquire effective information.

[0034] Step S202: Control the feature-fused image to be input into the target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine conveyor belt.

[0035] Specifically, in this application, the wavelet neural network is used to perform semantic segmentation on the multi-feature fused image obtained in the previous step, and a semantic segmentation image containing the contour information of the mine conveyor belt is obtained. Among them, the wavelet neural network has more efficient convergence ability and model fitting ability compared with other artificial neural networks.

[0036] Step S203: Repair the contour of the mine conveyor belt in the semantic segmentation image based on the contour repair algorithm to determine the complete contour corresponding to the mine conveyor belt.

[0037] Specifically, in this application, the contour repair algorithm is used to repair the contour of the mine conveyor belt in the semantic segmentation image. Since the contour of the mine conveyor belt in the semantic segmentation image output by the wavelet neural network is incomplete or unclear, the contour repair algorithm can be further used to repair the conveyor belt contour to obtain the complete contour of the mine conveyor belt.

[0038] Step S204: Determine the warning line in the semantic segmentation image. Based on the complete contour and the warning line, determine whether the mine conveyor belt has a deviation phenomenon, and trigger an alarm when the mine conveyor belt has a deviation phenomenon, where the warning line is two left and right lines composed of continuous pixel points.

[0039] Through this embodiment, by first considering various image features, and then using a wavelet neural network, which combines the time-frequency localization characteristics of wavelet transform and the self-learning characteristics of neural network, and has higher degrees of freedom, elasticity and plasticity compared with traditional RBF and BP networks. Therefore, it has good image approximation and pattern recognition and classification capabilities, and can ensure the effectiveness of subsequent feature extraction in the relatively complex real-time scene in the mine. Finally, a contour repair method based on contour information is introduced according to the shape characteristics of the conveyor belt, which refines the segmentation result of the target boundary, achieving the technical effect of further improving the image segmentation accuracy.

[0040] In the specific implementation process, the above step S201 can be specifically implemented through the following steps: Determine multiple image features, where the multiple image features at least include spatial color, edge information, and contrast. The spatial color at least includes the RGB original image, image hue, image saturation, and image brightness; Control the initial mine tunnel image to perform feature enhancement processing in multiple dimensions corresponding to the multiple image features respectively to obtain multiple groups of data corresponding to the multiple dimensions; Fuse the multiple groups of data to obtain a feature fusion image. The core of this step is to enhance the information of each image feature in the image, improve the acquisition of effective information by the subsequent semantic segmentation network, and reduce the influence of image noise on the network, thereby improving the recognition accuracy of the target. This step mainly enhances in three aspects: image color space, edge information, and contrast. In addition to the RGB original image, the color space also introduces the hue-saturation-value (HSV) color space, enabling the network to obtain target information from six color channels: red, green, blue, hue, saturation, and brightness, making the obtained feature dimensions higher; The edge information is enhanced by means of edge detection and image sharpening. Considering the detection efficiency of the algorithm, the Canny edge detection and Laplacian image sharpening algorithms are adopted; The enhancement of image contrast mainly uses the grayscale image histogram equalization algorithm to enhance the contrast of images with a small dynamic range by adjusting the gray levels of each pixel in the image, and to a certain extent reduce the influence of illumination on the image.

[0041] This application uses a wavelet neural network to perform semantic segmentation on images. Before the above step S202, the following steps are also included: Obtain an initial wavelet neural network, where the initial wavelet neural network consists of an input layer, a hidden layer, and an output layer. The initial wavelet neural network contains multiple wavelet elements, and the structure in the initial wavelet neural network is a compact structure. The compact structure integrates wavelet functions into the neural network; Through the gradient descent method, correct the network weights and the parameters of the wavelet functions in the initial wavelet neural network, and obtain the corrected wavelet neural network; Determine the corrected wavelet neural network as the target wavelet neural network. Among them, the wavelet neural network adopts the same topological structure as the artificial neural network, only using wavelet elements to replace neurons. The network topology structure diagram of the wavelet neural network is as Figure 3 shown. This structure integrates wavelet functions into the network and is called a compact structure.

[0042] Furthermore, Figure 3 the x1, x2…x n-1 , x n are the input samples of the input layer, is the Morlet wavelet function, and y1, y2…y m-1 , y m are the network outputs of the wavelet neural network.

[0043] When the input sample sequence is x i (i = 1, 2, … k), the output expression of the hidden layer is:

[0044]

[0045] In the formula, h(j) is the output of the hidden layer, w ij is the weight value between the input layer and the hidden layer, h j is the Morlet wavelet function, b j is the translation factor of the wavelet function, a j is the scaling factor of the wavelet function.

[0046] The Morlet wavelet function used is: h j = cos(1.75x) exp(-x 2 / 2)

[0047] The calculation formula for the output layer node is:

[0048]

[0049] The weight parameter correction algorithm of the wavelet neural network uses the gradient descent method to correct the weights of the network and the parameters of the wavelet function, so that the actual output of the network continuously approaches the expected output. The correction process of the wavelet neural network is as follows:

[0050] (1) Take the sum of squared errors as the objective function, and the mathematical expression is:

[0051]

[0052] In the above formula, d i p is the expected output value, y i p is the actual output value.

[0053] (2) Correct the weights of the network and the scaling and translation factors of the wavelet basis function according to the error E:

[0054]

[0055]

[0056]

[0057] Among them, η is the learning rate, and α (0 < α < 1) is the corresponding momentum factor.

[0058] The wavelet neural network adopted in this application combines the advantages of wavelet theory and artificial neural network, and has more efficient convergence ability and model fitting ability.

[0059] Step S203 specifically includes the following content:

[0060] Determine the contour pixel points in the semantic segmentation image through a contour repair algorithm, and repair the contour of the mine conveyor belt based on the contour pixel points to obtain a complete contour, where the contour pixel points are the pixel points corresponding to the contour of the mine conveyor belt.

[0061] Considering the shape characteristics of the conveyor belt, Step S203 adopts a contour repair method based on contour information and uses it to appropriately constrain and correct the contour of the initial segmentation result. Its main idea is to first determine the contour pixel points in the image, then connect these pixel points together to form the required regional contour curve, and finally use the formed contour curve to segment different category regions of the segmented image, so as to obtain the segmentation result with the contour adjusted.

[0062] Further, Step S203 is divided into the following sub-steps:

[0063] Step S2031: Traverse each pixel point in the semantic segmentation image in sequence, and respectively determine whether each pixel point is a contour pixel point;

[0064] Step S2032: If the pixel point is a contour pixel point, label the pixel point and determine all the labeled pixel points in the semantic segmentation image;

[0065] Step S2033: Connect all the labeled pixel points to obtain a connected curve, and repair the contour of the mine conveyor belt based on the connected curve.

[0066] In an optional embodiment, set a deviation warning line according to the actual situation of the conveyor belt. The warning line is two straight lines composed of continuous pixel points set in the image, and its position is the left and right edges of the conveyor belt in a reasonable working state. The pixels labeled as the conveyor belt in the segmentation map obtained in Step S204 will be judged. When the conveyor belt is in a normal working state, it should be between the two warning lines, and all the pixels labeled as the conveyor belt do not pass through the warning line. When the conveyor belt is in an abnormal working state, it will deviate to outside the warning line, and the pixels labeled as the conveyor belt will coincide with the pixels of the warning line. At this time, the deviation of the conveyor belt is detected and a deviation alarm is issued. Specifically, it includes the following steps:

[0067] S2041: Determine the contour pixel points corresponding to the complete contour and the warning line pixel points corresponding to the warning line;

[0068] S2042: When there is a coincidence between the contour pixel points and the warning line pixel points, it is determined that the mine conveyor belt has a deviation phenomenon.

[0069] In an optional embodiment, in step S201, real-time video of the conveyor belt operation is collected by a camera sensor installed above the coal mine conveyor belt. Each frame in the video is the image data of the conveyor belt operation sampled at fixed time intervals. To improve the stability of the algorithm operation, the camera sensor should be installed directly above the conveyor belt in its normal working state, and the two side edges of the conveyor belt can be reflected in the video image.

[0070] A method for detecting the deviation of a mine roadway conveyor belt provided by this application first considers a variety of traditional image features, and then uses a wavelet neural network, which combines the time-frequency localization characteristics of wavelet transform and the self-learning characteristics of neural network. Compared with traditional RBF and BP networks, it has higher degrees of freedom, elasticity and plasticity. Therefore, it has good image approximation and pattern recognition and classification capabilities, and can ensure the effectiveness of subsequent feature extraction in the relatively complex real-time scenarios in the mine roadway. Finally, a contour repair method based on contour information is introduced according to the shape characteristics of the conveyor belt, which refines the segmentation result of the target boundary, further improves the segmentation accuracy, and solves the technical problems in the prior art that the traditional image segmentation method has high efficiency but low accuracy, while the deep learning semantic segmentation method has high accuracy but poor efficiency and interpretability.

[0071] The embodiment of this application also provides a device for detecting the deviation of a mine roadway conveyor belt. It should be noted that the device for detecting the deviation of a mine roadway conveyor belt in the embodiment of this application can be used to execute the method for detecting the deviation of a mine roadway conveyor belt provided by the embodiment of this application. The device is used to implement the above-mentioned embodiment and preferred implementation manner, and those that have been described will not be repeated. As used hereinafter, the term "module" can be a combination of software and / or hardware that can achieve a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.

[0072] The following introduces a device for detecting the deviation of a mine roadway conveyor belt provided by the embodiment of this application.

[0073] Figure 4 is a schematic diagram of a device for detecting the deviation of a mine roadway conveyor belt provided by the embodiment of this application. As Figure 4As shown in the figure, the device includes: a first acquisition unit 401, configured to acquire an initial mine tunnel image, perform multi-feature enhancement processing on the initial mine tunnel image, and acquire a feature fusion image; a first control unit 402, configured to control the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine tunnel conveyor belt; a first determination unit 403, configured to repair the contour of the mine tunnel conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine tunnel conveyor belt; a second determination unit 404, configured to determine the warning line in the semantic segmentation image, and based on the complete contour and the warning line, determine whether the mine tunnel conveyor belt has a deviation phenomenon, and trigger an alarm when the mine tunnel conveyor belt has a deviation phenomenon, where the warning line is two left and right lines composed of continuous pixel points.

[0074] As an optional solution, the first acquisition unit 401 includes: a first determination subunit, configured to determine multiple image features, where the multiple image features at least include spatial color, edge information, and contrast, and the spatial color at least includes the RGB original image, image hue, image saturation, and image lightness; a control subunit, configured to control the initial mine tunnel image to perform feature enhancement processing in multiple dimensions corresponding to the multiple image features to obtain multiple groups of data corresponding to the multiple dimensions; a fusion subunit, configured to fuse the multiple groups of data to obtain a feature fusion image.

[0075] An optional solution, the device includes: a second acquisition unit, configured to acquire an initial wavelet neural network before controlling the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the initial wavelet neural network is composed of an input layer, a hidden layer, and an output layer, the initial wavelet neural network contains multiple wavelet elements, and the structure in the initial wavelet neural network is a compact structure, and the compact structure fuses the wavelet function in the neural network; a correction unit, configured to correct the network weights and the parameters of the wavelet function in the initial wavelet neural network by the gradient descent method, and obtain a corrected wavelet neural network; a third determination unit, configured to determine the corrected wavelet neural network as the target wavelet neural network.

[0076] An optional solution, the first determination unit 403 includes: a repair subunit, configured to determine the contour pixel points in the semantic segmentation image through a contour repair algorithm, and repair the contour of the mine tunnel conveyor belt according to the contour pixel points to obtain a complete contour, where the contour pixel points are the pixel points corresponding to the contour of the mine tunnel conveyor belt.

[0077] An alternative solution, the repair subunit includes: a judgment module, configured to sequentially traverse each pixel point in the semantic segmentation image and respectively judge whether each pixel point is a contour pixel point; a determination module, configured to, when the pixel point is a contour pixel point, label the pixel point and determine all the labeled pixel points in the semantic segmentation image; a repair module, configured to connect all the labeled pixel points to obtain a connected curve, and repair the contour of the mine belt conveyor according to the connected curve.

[0078] An alternative solution, the second determination unit 404 includes: a second determination subunit, configured to determine the contour pixel points corresponding to the complete contour and the warning line pixel points corresponding to the warning line; a third determination subunit, configured to determine that the mine belt conveyor has a deviation phenomenon when there is an overlapping phenomenon between the contour pixel points and the warning line pixel points.

[0079] An alternative solution, the first acquisition unit 401 includes: an acquisition subunit, configured to collect the working video of the mine roadway and obtain the initial mine roadway image according to the working video.

[0080] A detection device for mine belt conveyor deviation includes a processor and a memory. The above first acquisition unit 401 etc. are all stored in the memory as program units, and the corresponding functions are realized by the processor executing the above program units stored in the memory. The above modules are all located in the same processor; or, the above each module is located in different processors in any combination form.

[0081] The processor contains a kernel, and the kernel is used to retrieve the corresponding program units from the memory. One or more kernels can be set, and by adjusting the kernel parameters, the technical problems in the prior art that the traditional image segmentation method has high efficiency but low accuracy, and the semantic segmentation method of deep learning has high accuracy but poor efficiency and interpretability are solved.

[0082] The memory may include non-permanent memory in computer-readable media, random access memory (RAM) and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM), and the memory includes at least one storage chip.

[0083] An embodiment of the present invention provides a computer-readable storage medium, and the computer-readable storage medium includes a stored program, wherein when the program runs, it controls the device where the computer-readable storage medium is located to execute a detection method for mine belt conveyor deviation.

[0084] Specifically, a detection method for mine belt conveyor deviation includes:

[0085] Step S201, obtain the initial mine roadway image, perform multi-feature enhancement processing on the initial mine roadway image, and obtain the feature fusion image;

[0086] Specifically, image data of the mine conveyor belt is collected by a camera sensor installed above the coal mine conveyor belt. In this application, by enhancing the mine tunnel image in multiple feature dimensions, a fused image with multiple features is obtained, thereby improving the acquisition of effective information by the subsequent semantic segmentation network.

[0087] Step S202: Control the feature-fused image to be input into the target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine conveyor belt.

[0088] Specifically, in this application, the wavelet neural network performs semantic segmentation on the multi-feature fused image obtained in the previous step and obtains a semantic segmentation image containing the contour information of the mine conveyor belt. Among them, the wavelet neural network has more efficient convergence ability and model fitting ability compared with other artificial neural networks.

[0089] Step S203: Repair the contour of the mine conveyor belt in the semantic segmentation image based on the contour repair algorithm to determine the complete contour corresponding to the mine conveyor belt.

[0090] Specifically, in this application, the contour repair algorithm is used to repair the contour of the mine conveyor belt in the semantic segmentation image. Since the contour of the mine conveyor belt in the semantic segmentation image output by the wavelet neural network is incomplete or unclear, the contour repair algorithm can be used to further repair the conveyor belt contour to obtain the complete contour of the mine conveyor belt.

[0091] Step S204: Determine the warning line in the semantic segmentation image. Based on the complete contour and the warning line, determine whether the mine conveyor belt has a deviation phenomenon, and trigger an alarm when the mine conveyor belt has a deviation phenomenon, where the warning line is two left and right lines composed of continuous pixel points.

[0092] Optionally, obtain the initial mine tunnel image, perform multi-feature enhancement processing on the initial mine tunnel image, and obtain the feature-fused image.

[0093] Optionally, control the feature-fused image to be input into the target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine conveyor belt.

[0094] Optionally, repair the contour of the mine conveyor belt in the semantic segmentation image based on the contour repair algorithm to determine the complete contour corresponding to the mine conveyor belt.

[0095] Optionally, determine the warning line in the semantic segmentation image. Based on the complete contour and the warning line, determine whether the mine belt conveyor is running off track. If the mine belt conveyor is running off track, trigger an alarm. The warning line is two lines composed of continuous pixel points on the left and right.

[0096] An embodiment of the present invention provides a processor for running a program. When the program runs, it executes a method for detecting whether a mine belt conveyor is running off track.

[0097] Specifically, the method includes:

[0098] Step S201: Obtain an initial mine image, perform multi-feature enhancement processing on the initial mine image, and obtain a feature fusion image.

[0099] Specifically, collect the image data of the mine belt conveyor through a camera sensor installed above the coal mine conveyor belt. In this application, by enhancing multiple features of the mine image in multiple dimensions, a feature fusion image is obtained, thereby improving the acquisition of effective information by the subsequent semantic segmentation network.

[0100] Step S202: Control the feature fusion image to be input into the target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network. The semantic segmentation image includes the contour of the mine belt conveyor.

[0101] Specifically, in this application, the wavelet neural network is used to perform semantic segmentation on the multi-feature fusion image obtained in the previous step, and a semantic segmentation image containing the contour information of the mine belt conveyor is obtained. The wavelet neural network has more efficient convergence ability and model fitting ability compared to other artificial neural networks.

[0102] Step S203: Repair the contour of the mine belt conveyor in the semantic segmentation image based on the contour repair algorithm to determine the complete contour corresponding to the mine belt conveyor.

[0103] Specifically, in this application, the contour repair algorithm is used to repair the contour of the mine belt conveyor in the semantic segmentation image. Since the contour of the mine belt conveyor in the semantic segmentation image output by the wavelet neural network is incomplete or unclear, the contour repair algorithm can be further used to repair the conveyor belt contour to obtain the complete contour of the mine belt conveyor.

[0104] Step S204: Determine the warning line in the semantic segmentation image. Based on the complete contour and the warning line, determine whether the mine belt conveyor is running off track. If the mine belt conveyor is running off track, trigger an alarm. The warning line is two lines composed of continuous pixel points on the left and right.

[0105] Optionally, obtain an initial mine roadway image, perform multi-feature enhancement processing on the initial mine roadway image, and obtain a feature fusion image.

[0106] Optionally, control the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine roadway conveyor belt.

[0107] Optionally, repair the contour of the mine roadway conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine roadway conveyor belt.

[0108] Optionally, determine the warning line in the semantic segmentation image, and based on the complete contour and the warning line, determine whether the mine roadway conveyor belt has a deviation phenomenon, and trigger an alarm when the mine roadway conveyor belt has a deviation phenomenon, where the warning line is two left and right lines composed of continuous pixel points.

[0109] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored on the memory and executable on the processor. When the processor executes the program, it implements at least the following steps: Step S201, obtain an initial mine roadway image, perform multi-feature enhancement processing on the initial mine roadway image, and obtain a feature fusion image;

[0110] Specifically, the image data of the mine roadway conveyor belt is collected by a camera sensor installed above the coal mine conveyor belt. In this application, by enhancing multiple features of the mine roadway image in multiple dimensions, a feature fusion image is obtained, thereby improving the acquisition of effective information by the subsequent semantic segmentation network.

[0111] Step S202, control the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine roadway conveyor belt;

[0112] Specifically, in this application, the wavelet neural network performs semantic segmentation on the multi-feature fusion image obtained in the previous step and obtains a semantic segmentation image containing the contour information of the mine roadway conveyor belt. Among them, the wavelet neural network has more efficient convergence ability and model fitting ability compared with other artificial neural networks.

[0113] Step S203, repair the contour of the mine roadway conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine roadway conveyor belt;

[0114] Specifically, in the present application, the contour of the mine conveyor belt in the semantic segmentation image is repaired by the contour repair algorithm. Since the contour of the mine conveyor belt in the semantic segmentation image output by the wavelet neural network is incomplete or unclear, the contour of the conveyor belt can be further repaired by the contour repair algorithm to obtain the complete contour of the mine conveyor belt.

[0115] Step S204: Determine the warning line in the semantic segmentation image. Based on the complete contour and the warning line, determine whether the mine conveyor belt is running off track, and trigger an alarm when the mine conveyor belt is running off track. Here, the warning line is two left and right lines composed of continuous pixel points.

[0116] Optionally, obtain the initial mine image, perform multi-feature enhancement processing on the initial mine image, and obtain the feature fusion image.

[0117] Optionally, control the feature fusion image to be input into the target wavelet neural network to obtain the semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine conveyor belt.

[0118] Optionally, repair the contour of the mine conveyor belt in the semantic segmentation image based on the contour repair algorithm to determine the complete contour corresponding to the mine conveyor belt.

[0119] Optionally, determine the warning line in the semantic segmentation image. Based on the complete contour and the warning line, determine whether the mine conveyor belt is running off track, and trigger an alarm when the mine conveyor belt is running off track. Here, the warning line is two left and right lines composed of continuous pixel points. The device in this article can be a server, a PC, a PAD, a mobile phone, etc.

[0120] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program initialized with at least the following method steps: obtaining the initial mine image, performing multi-feature enhancement processing on the initial mine image, and obtaining the feature fusion image; controlling the feature fusion image to be input into the target wavelet neural network to obtain the semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine conveyor belt; repairing the contour of the mine conveyor belt in the semantic segmentation image based on the contour repair algorithm to determine the complete contour corresponding to the mine conveyor belt; determining the warning line in the semantic segmentation image, and based on the complete contour and the warning line, determining whether the mine conveyor belt is running off track, and triggering an alarm when the mine conveyor belt is running off track. Here, the warning line is two left and right lines composed of continuous pixel points.

[0121] Optionally, perform multi-feature enhancement processing on the initial mine roadway image and obtain a feature fusion image, including: determining multiple image features, where the multiple image features at least include spatial color, edge information, and contrast, and the spatial color at least includes the RGB original image, image hue, image saturation, and image brightness; controlling the initial mine roadway image to perform feature enhancement processing in multiple dimensions corresponding to the multiple image features to obtain multiple groups of data corresponding to the multiple dimensions; and fusing the multiple groups of data to obtain a feature fusion image.

[0122] Optionally, before controlling the feature fusion image to be input into the target wavelet neural network to obtain the semantic segmentation image output by the target wavelet neural network, the method includes: obtaining an initial wavelet neural network, where the initial wavelet neural network is composed of an input layer, a hidden layer, and an output layer, the initial wavelet neural network contains multiple wavelet elements, and the structure of the initial wavelet neural network is a compact structure, and the compact structure fuses wavelet functions in the neural network; correcting the network weights and the parameters of the wavelet functions in the initial wavelet neural network by the gradient descent method, and obtaining a corrected wavelet neural network; and determining the corrected wavelet neural network as the target wavelet neural network.

[0123] Optionally, repair the contour of the mine roadway conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine roadway conveyor belt, including: determining the contour pixel points in the semantic segmentation image by the contour repair algorithm, and repairing the contour of the mine roadway conveyor belt based on the contour pixel points to obtain a complete contour, where the contour pixel points are the pixel points corresponding to the contour of the mine roadway conveyor belt.

[0124] Optionally, determine the contour pixel points in the semantic segmentation image by the contour repair algorithm, and repair the contour of the mine roadway conveyor belt based on the contour pixel points, including: sequentially traversing each pixel point in the semantic segmentation image, and respectively determining whether each pixel point is a contour pixel point; if the pixel point is a contour pixel point, then label the pixel point, and determine all the labeled pixel points in the semantic segmentation image; connect all the labeled pixel points to obtain a connected curve, and repair the contour of the mine roadway conveyor belt based on the connected curve.

[0125] Optionally, determine whether the mine roadway conveyor belt has a deviation phenomenon based on the complete contour and the warning line, including: determining the contour pixel points corresponding to the complete contour and the warning line pixel points corresponding to the warning line; when there is an overlapping phenomenon between the contour pixel points and the warning line pixel points, determining that the mine roadway conveyor belt has a deviation phenomenon.

[0126] Optionally, obtain an initial mine roadway image, including: collecting the working video of the mine roadway, and obtaining the initial mine roadway image based on the working video.

[0127] Obviously, those skilled in the art should understand that the various modules or steps of the present invention described above can be implemented by a general-purpose computing device. They can be concentrated on a single computing device or distributed over a network composed of multiple computing devices. They can be implemented by program codes executable by the computing device. Thus, they can be stored in a storage device and executed by the computing device. And in some cases, the steps shown or described can be executed in a different order than here, or they can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. In this way, the present invention is not limited to any specific combination of hardware and software.

[0128] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0129] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0130] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process. Thus, the instructions executed on the computer or other programmable device provide for implementing the functions in the processFigure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0132] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0133] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0134] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0135] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0136] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0137] 1) The detection method of this application combines traditional segmentation technology and deep learning technology. It enhances the key information of the image through traditional machine vision processing methods such as color space conversion, gray level equalization, edge detection, and image sharpening, and then uses a wavelet neural network to extract target features for semantic segmentation. At the same time, contour repair is combined to refine the boundaries of each contour in the image, mainly the boundary of the conveyor belt. Finally, through efficient segmentation, the position of the conveyor belt is located to achieve the deviation detection of the conveyor belt.

[0138] 2) This application uses a wavelet neural network, which is compatible with the time-frequency localization characteristics of wavelet transform and the self-learning characteristics of neural networks. Compared with traditional RBF and BP networks, it has higher degrees of freedom, elasticity, and plasticity. Therefore, it has good image approximation and pattern recognition and classification capabilities, and can ensure the effectiveness of subsequent feature extraction in relatively complex real-time scenarios in the mine roadway.

[0139] The above are only the preferred embodiments of this application and are not used to limit this application. For those skilled in the art, this application can have various changes and modifications. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of this application shall be included within the protection scope of this application.

Claims

1. A detection method for the deviation of a mine conveyor belt, characterized in that, Including: Obtain an initial mine tunnel image, perform multi-feature enhancement processing on the initial mine tunnel image, and obtain a feature fusion image; Control the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of a mine tunnel conveyor belt; Based on a contour repair algorithm, repair the contour of the mine tunnel conveyor belt in the semantic segmentation image to determine the complete contour corresponding to the mine tunnel conveyor belt; Determine the warning line in the semantic segmentation image, and based on the complete contour and the warning line, determine whether the mine tunnel conveyor belt has a deviation phenomenon, and trigger an alarm when the mine tunnel conveyor belt has the deviation phenomenon, where the warning line is two left and right lines composed of continuous pixel points; Wherein, before controlling the feature fusion image to be input into a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, the method includes: obtaining an initial wavelet neural network, where the initial wavelet neural network is composed of an input layer, a hidden layer, and an output layer, the initial wavelet neural network contains multiple wavelet elements, the structure in the initial wavelet neural network is a compact structure, and the compact structure fuses wavelet functions in the neural network; by using the gradient descent method, correct the network weights and the parameters of the wavelet function in the initial wavelet neural network, and obtain a corrected wavelet neural network; determine the corrected wavelet neural network as the target wavelet neural network; Based on a contour repair algorithm, repair the contour of the mine tunnel conveyor belt in the semantic segmentation image to determine the complete contour corresponding to the mine tunnel conveyor belt, including: determining the contour pixel points in the semantic segmentation image through the contour repair algorithm, and repairing the contour of the mine tunnel conveyor belt based on the contour pixel points to obtain the complete contour, where the contour pixel points are the pixel points corresponding to the contour of the mine tunnel conveyor belt.

2. The method according to claim 1, wherein Performing multi-feature enhancement processing on the initial mine tunnel image and obtaining a feature fusion image includes: Determine multiple image features, where the multiple image features at least include spatial color, edge information, and contrast, and the spatial color at least includes an RGB original image, image hue, image saturation, and image brightness; Control the initial mine tunnel image to perform feature enhancement processing in multiple dimensions corresponding to the multiple image features to obtain multiple groups of data corresponding to the multiple dimensions; Fuse the multiple groups of data to obtain the feature fusion image.

3. The method according to claim 1, wherein Determining the contour pixel points in the semantic segmentation image through the contour repair algorithm and repairing the contour of the mine tunnel conveyor belt based on the contour pixel points includes: Traverse each pixel point in the semantic segmentation image in sequence, and respectively determine whether each pixel point is a contour pixel point; If the pixel point is a contour pixel point, label the pixel point and determine all the labeled pixel points in the semantic segmentation image; Connect all the marked pixel points to obtain a connected curve, and repair the contour of the mine conveyor belt according to the connected curve.

4. The method according to claim 1, wherein Determine whether the mine conveyor belt is running off track based on the complete contour and the warning line, including: Determine the contour pixel points corresponding to the complete contour and the warning line pixel points corresponding to the warning line; When there is an overlapping phenomenon between the contour pixel points and the warning line pixel points, determine that the mine conveyor belt has the running-off-track phenomenon.

5. The method according to claim 1, characterized in that Obtain the initial mine image, including: Collect the working video of the mine, and obtain the initial mine image according to the working video.

6. A detection device for belt deviation of a mine roadway, characterized in that, Including: A first acquisition unit for obtaining an initial mine image, performing multi-feature enhancement processing on the initial mine image, and obtaining a feature fusion image; A first control unit for controlling the input of the feature fusion image to a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network, where the semantic segmentation image includes the contour of the mine conveyor belt; A first determination unit for repairing the contour of the mine conveyor belt in the semantic segmentation image based on a contour repair algorithm to determine the complete contour corresponding to the mine conveyor belt; A second determination unit for determining the warning line in the semantic segmentation image, determining whether the mine conveyor belt is running off track based on the complete contour and the warning line, and triggering an alarm when the mine conveyor belt has the running-off-track phenomenon, where the warning line is two left and right lines composed of continuous pixel points; The device further includes a second acquisition unit for obtaining an initial wavelet neural network before controlling the input of the feature fusion image to a target wavelet neural network to obtain a semantic segmentation image output by the target wavelet neural network. The initial wavelet neural network is composed of an input layer, a hidden layer, and an output layer. The initial wavelet neural network contains multiple wavelet elements, and the structure in the initial wavelet neural network is a compact structure, and the compact structure fuses wavelet functions in the neural network; a correction unit for correcting the network weights and the parameters of the wavelet function in the initial wavelet neural network by the gradient descent method and obtaining a corrected wavelet neural network; a third determination unit for determining the corrected wavelet neural network as the target wavelet neural network; The first determination unit includes: a repair subunit for determining the contour pixel points in the semantic segmentation image through a contour repair algorithm, and repairing the contour of the mine conveyor belt according to the contour pixel points to obtain a complete contour, where the contour pixel points are the pixel points corresponding to the contour of the mine conveyor belt.

7. A processor, characterized in that, The processor is used to run a program, where the program, when running, executes a method for detecting the running-off-track of a mine conveyor belt according to any one of claims 1 to 5.

8. An electronic device, characterized in that, Including: One or more processors, a memory, and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by the one or more processors. The one or more programs include a method for detecting the running-off-track of a mine conveyor belt according to any one of claims 1 to 5.

Images