Intelligent monitoring and detection method, system and electronic equipment for belt conveyor

Abstract

The application provides an intelligent monitoring and detection method and system for a belt conveyor and an electronic device. The method can detect material transmission data of the belt conveyor by using a monitoring sensor. The material transmission data is multidimensional data. Thus, the material transmission data can be subjected to foreign matter identification, detection of belt deviation, detection of material accumulation, detection of material granularity, detection of longitudinal tearing, monitoring of material flow, and fusion monitoring of carrier roller faults. Corresponding monitoring results of the various detections are obtained. The belt conveyor can timely detect faults or abnormalities through the various detections, and timely processing is performed, thereby ensuring normal operation of the belt conveyor.

Description

Technical Field

[0001] This application relates to the field of data detection technology, and in particular to an intelligent monitoring and detection method, system and electronic equipment for belt conveyors. Background Technology

[0002] Belt conveyors, especially high-capacity, long-distance belt conveyors, are prone to operational abnormalities or malfunctions during use.

[0003] Therefore, in order to ensure that belt conveyors can operate normally, they need to be inspected. However, the current inspection methods for belt conveyors are relatively simple and not comprehensive enough, which makes the reliability of belt conveyor inspections worse. Summary of the Invention

[0004] In view of this, the purpose of this application is to propose an intelligent monitoring and detection method, system and electronic equipment for belt conveyors to solve or partially solve the above-mentioned technical problems.

[0005] To achieve the above objectives, this application provides an intelligent monitoring and detection method for belt conveyors, characterized by comprising:

[0006] Utilize monitoring sensors to detect material transfer data of belt conveyors;

[0007] Based on the material conveying video in the material transmission data, foreign object identification processing is performed, and the result of foreign object identification processing is determined.

[0008] Based on the conveyor belt running image in the material transfer data, conveyor belt misalignment detection is performed to obtain the conveyor belt misalignment detection result;

[0009] Based on the image of the material discharge port in the material transfer data, material accumulation detection is performed to obtain the material accumulation detection result;

[0010] Based on the material image of the conveyor belt in the material transmission data, the particle size of the material is detected to obtain the particle size detection result.

[0011] Based on the conveyor belt surface image in the material transfer data, longitudinal tear detection is performed to obtain the longitudinal tear detection result;

[0012] Based on the point cloud data of the material cross-section of the conveyor belt in the material transfer data, material flow rate is monitored to obtain the material flow rate result;

[0013] Based on the operating audio and thermal infrared images of the conveyor belt idlers in the material transfer data, the operating audio and thermal infrared images are fused together for idler fault monitoring to obtain idler fault monitoring results.

[0014] Based on the same inventive concept, this application also provides an intelligent monitoring and detection system for a belt conveyor, comprising:

[0015] The data detection module is configured to detect the material transfer data of the belt conveyor using monitoring sensors;

[0016] The foreign object identification module is configured to perform foreign object identification processing based on the material conveying video in the material transmission data, and determine the foreign object identification processing result;

[0017] The belt misalignment detection module is configured to perform conveyor belt misalignment detection based on the conveyor belt running image in the material transmission data, and obtain the conveyor belt misalignment detection result;

[0018] The material accumulation detection module is configured to perform material accumulation detection based on the material discharge port image in the material transmission data, and obtain the material accumulation detection result.

[0019] The particle size detection module is configured to perform particle size detection based on the conveyor belt material image in the material transmission data, and obtain the particle size detection result.

[0020] The longitudinal tear detection module is configured to perform longitudinal tear detection based on the conveyor belt surface image in the material transport data, and obtain the longitudinal tear detection result;

[0021] The material flow detection module is configured to monitor the material flow based on the point cloud data of the cross-section of the material in the conveyor belt in the material transmission data, and obtain the material flow result.

[0022] The idler roller fault detection module is configured to perform idler roller fault fusion monitoring based on the operating audio and thermal infrared images of the conveyor belt idlers in the material transfer data, and obtain idler roller fault monitoring results.

[0023] Based on the same inventive concept, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable by the processor, wherein the processor implements the method described above when executing the computer program.

[0024] As can be seen from the above, the intelligent monitoring and detection method, system, and electronic equipment for belt conveyors provided in this application can utilize monitoring sensors to detect the material transmission data of the belt conveyor. This material transmission data is multi-dimensional, enabling foreign object identification processing on the material transmission video in the material transmission data, thereby obtaining accurate foreign object identification processing results; it can also perform conveyor belt deviation detection on the conveyor belt running image in the material transmission data, thereby obtaining accurate conveyor belt deviation detection results; it can also perform material accumulation detection on the material drop port image in the material transmission data, thereby obtaining accurate material accumulation detection results; it can also perform material particle size detection on the material image of the conveyor belt in the material transmission data, thereby obtaining accurate material particle size detection results; it can also perform longitudinal tear detection on the surface image of the conveyor belt in the material transmission data, thereby obtaining accurate longitudinal tear detection results; it can also perform material flow monitoring on the point cloud data of the cross-section of the material in the material transmission data, thereby obtaining accurate material flow results; and it can also perform idler fault fusion monitoring on the running audio and thermal infrared images of the conveyor belt idlers in the material transmission data, thereby obtaining idler fault monitoring results. This allows for multi-faceted detection of the belt conveyor, including foreign object identification, conveyor belt misalignment, material accumulation, material particle size, longitudinal tearing, material flow rate, and idler roller malfunctions. This ensures that the belt conveyor can detect faults or anomalies in a timely manner through multi-faceted detection, and thus take timely action to ensure the normal operation of the belt conveyor. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of an intelligent monitoring and detection method for a belt conveyor according to an embodiment of this application;

[0027] Figure 2 Examples of embodiments of this application Schematic diagram of the calculation principle;

[0028] Figure 3 The trapezoidal element area integration method is used to calculate the area of ​​this application embodiment. Schematic diagram;

[0029] Figure 4 Examples of embodiments of this application Schematic diagram of the calculation principle;

[0030] Figure 5This is a schematic diagram of the intelligent monitoring and detection system for a belt conveyor according to an embodiment of this application;

[0031] Figure 6 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with specific embodiments and the accompanying drawings.

[0033] It should be noted that, unless otherwise defined, the technical or scientific terms used in the embodiments of this application should have the ordinary meaning understood by one of ordinary skill in the art to which this application pertains. The terms "first," "second," and similar terms used in the embodiments of this application do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are only used to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0034] Definitions:

[0035] YOLOv7: You Only Look Once version 7 is an end-to-end object detection algorithm that identifies and locates multiple target objects from images or videos.

[0036] RepVGG block: is the basic building block of RepVGG network.

[0037] CBS: In the YOLO series of object detection models, the CBS layer is a basic convolutional module consisting of three consecutive operations: a two-dimensional convolutional layer (Conv), a batch normalization layer (BN), and a SiLU activation function (Sigmoid LinearUnit).

[0038] ELAN: Eficient Layer Aggregation Networks, is an important network structure in the YOLO series of object detection algorithms. It aims to improve the accuracy and robustness of object detection algorithms by effectively aggregating feature information from different layers.

[0039] MP: In the YOLOv7 network architecture, the MP layer is a basic component of the backbone network and is mainly responsible for downsampling operations.

[0040] CNN: Convolutional Neural Network.

[0041] ROI: Region of Interest detection.

[0042] In related technologies, belt conveyors are a type of continuous transportation equipment used in production. They are mainly composed of conveyor belts, frames, idlers, drums, tensioning devices, and transmission devices. Among them, the conveyor belt is the key component for traction and load-bearing of belt conveyors. It has advantages such as large carrying capacity, long conveying distance, low energy consumption, low freight cost, high efficiency, stable operation, convenient loading and unloading, and suitability for bulk material transportation.

[0043] Belt conveyors, especially high-capacity, long-distance belt conveyors, are prone to failure. One type of failure is steel wire rope core failure, including steel wire rope breakage, corrosion, improper joint overlap, and pull, which are the main causes of belt breakage. The other type is surface failure, mainly including deviation, longitudinal tearing, surface damage, as well as related material blockage and foreign objects, which are frequent failures and key failures to prevent and control.

[0044] The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0045] The intelligent monitoring and detection method for belt conveyors proposed in this application is applied to the monitoring system of belt conveyors.

[0046] like Figure 1 As shown, the method includes:

[0047] Step 101: Use monitoring sensors to detect the material transfer data of the belt conveyor.

[0048] In practice, various monitoring sensors are installed at different locations on the belt conveyor, including infrared sensors, electromagnetic sensors, ion sensors, radar point cloud sensors, image sensors, and thermal infrared image sensors.

[0049] These monitoring sensors are used to detect material transfer data during the conveying process of the belt conveyor. The material transfer data is sent to the monitoring system for analysis and processing to determine the operating status of the material transfer of the belt conveyor. Once an abnormality or fault is detected, an alarm will be triggered or the belt conveyor will be stopped. The monitoring screen corresponding to the abnormality or fault will also pop up, showing the location of the abnormality or fault, as well as the nature and cause of the fault.

[0050] In addition, the monitoring system is equipped with a telephone, which can transmit the monitoring results (including foreign object identification and processing results, conveyor belt misalignment detection results, material accumulation detection results, material particle size detection results, longitudinal tear detection results, material flow rate results, and idler roller fault monitoring results) through broadcast, call, unicast, or group broadcast.

[0051] Telephones are an important component of monitoring systems. Convenient, reliable, and fully functional telephones are crucial for communicating monitoring results along the belt conveyor line, and are also essential for ensuring the safe operation of the belt conveyor and improving the economic benefits of enterprises.

[0052] The telephone has the following functions:

[0053] (1) Full-duplex amplified voice communication:

[0054] After a telephone is connected to the network, its IP address, number, and group can be configured through the monitoring system's host computer software.

[0055] When the telephone is not in "broadcast" or "alarm" mode, calls can be made to any terminal along the conveyor belt using the dial pad. Once the called terminal answers, the two terminals can have a full-duplex amplified call, achieving the "one-call" function. If a group number is dialed, all telephones within the called group will be called, achieving the "group call" function once the other party answers.

[0056] (2) Public address system broadcasting:

[0057] Under normal operating conditions, the telephones are in a waiting state. If the staff in the monitoring room need to broadcast along the conveyor belt, they can use the "broadcast" button on the monitoring system's host computer software to broadcast to all telephones along the conveyor belt. At this time, the telephones are in a one-way data receiving "broadcast" state, and the voice data sent by the host computer of the monitoring system will be sent to all telephones and played through the speakers.

[0058] (3) Voice alarm:

[0059] Even in standby mode, telephones can receive commands from the monitoring system's host computer. When the host computer detects a fault in the conveyor belt, it sends a corresponding voice alarm signal to the telephones along the conveyor belt. At this time, the telephones will enter "alarm" mode. Upon receiving the voice alarm signal, all telephones along the conveyor belt will play the pre-stored voice alarm corresponding to the fault through their speakers. This allows workers along the conveyor belt to know its working status and promptly control and repair it, avoiding personal injury and property damage.

[0060] The following section will elaborate on the specific monitoring process:

[0061] Step 102: Based on the material conveying video in the material transmission data, perform foreign object identification processing and determine the foreign object identification processing result.

[0062] In practice, an image sensor (e.g., a camera) is positioned approximately 15 meters behind the material drop point on the conveyor belt to capture video of material transport on the upper surface of the conveyor belt. This allows the monitoring system to utilize a pre-trained foreign object detection model (e.g., YOLOv7). This model can then be used to perform foreign object identification processing on the material transport video, resulting in a foreign object identification output.

[0063] YOLOv7, as a duty detection model, consists of three parts: an input layer, a backbone network, and a head layer.

[0064] First, the input image is preprocessed and aligned to a 640×640 RGB image, which is then input into the backbone network. Based on the three layers of output in the backbone network, the head layer continues to output three feature maps of different sizes through the backbone network. After passing through the RepVGG block and conv (convolutional layer), foreign objects are detected, and the final foreign object recognition result is output.

[0065] In the backbone network of the YOLOv7 model, after four CBSs, an ELAN is connected, followed by three MP + ELAN outputs with sizes of 80×80×512, 40×40×1024, and 20×20×1024 respectively. Each MP has 5 layers, and the ELAN has 8 layers, for a total of 51 layers in the backbone. The head part of YOLOv7 is similar to the structure of PAFPN, the same as YOLOv4 and YOLOv5. First, the 32x downsampled feature map C5 at the final output of the backbone is processed through SPPCSP, reducing the number of channels from 1024 to 512.

[0066] The YOLOv7 backbone feature extraction network is a CNN network. CNN has translation invariance and locality, but lacks the ability to model globally and long distances. Based on this, the Transformer framework from the field of natural language processing is introduced to build a CNN+Transformer architecture, forming the COTN module, which improves the target detection effect and enhances the detection effect of small gangue blocks, anchor bolts and irregularly shaped metals, thereby adapting to the environment of a large number of coal blocks on the conveyor belt.

[0067] The introduced Transformer is driven by a novel spatial modeling mechanism based on dot product self-attention, performing recursive gated convolutions. Through gated convolutions and recursive design, it executes high-order spatial interactions, giving it a high degree of flexibility and customizability. Since the YOLOv7 model has a large number of parameters and a large computational cost, the CNN+Transformer can be compatible with various convolution variants and extend the second-order interactions in self-attention to arbitrary orders without introducing additional computation.

[0068] A well-trained foreign object detection model can identify materials and determine the types of various materials being transported on the conveyor belt. Based on the identified material types, it can accurately identify foreign objects and output the outline and category of the foreign object as the identification result, achieving high accuracy. Examples include normal coal materials, or unusual materials such as gangue and I-beams.

[0069] Step 103: Based on the conveyor belt running image in the material transfer data, perform conveyor belt misalignment detection to obtain the conveyor belt misalignment detection result.

[0070] In practice, the image sensor is set above the conveyor belt and can capture images of the conveyor belt running from the side (including the edge of the conveyor belt and the idlers). The monitoring system is used to detect conveyor belt deviation from the belt and output accurate conveyor belt deviation detection results.

[0071] The monitoring system combines deep learning and machine vision, and consists of two stages: Stage 1 extracts the region of interest from the conveyor belt running image, and Stage 2 detects the edge of the conveyor belt and the outer edge of the idler roller and estimates the deviation based on their corrected coordinates.

[0072] Phase 1 consists of the following three steps: (1) The continuously acquired images of the conveyor belt running are cropped into two parts, upper and lower, named ImgO and ImgI; (2) ImgO and ImgI are input into a pre-trained ROI detector to extract the region of interest (ROI) containing the images of the conveyor belt edge and the outer edge of the idler roller; (3) If the region of interest (ROI) is detected, proceed to Phase 2.

[0073] The pre-trained ROI detector includes an OM-SSD feature extraction network, specifically:

[0074] The OM-SSD feature extraction network has 22 layers, as shown in Table 1. All convolutional layers (Conv) in Table 1 are followed by batch normalization and ReLU activation functions, including depthwise convolution and pointwise convolution in DSC.

[0075] Table 1. Configuration of the Feature Extraction Network for OM-SSD

[0076]

[0077] To accommodate targets with varying sizes in the predicted regions of interest (ROIs), OM-SSD uses six feature layers (L11, L13, L15, L17, L19, and L21) as prediction source layers when predicting the bounding boxes of the ROIs. For the k-th layer among these six layers, the ratio of the default bounding box to the input image size is... Determined by the following formula:

[0078] (1).

[0079] in, =0.2, =0.95 represents the maximum and minimum size ratios, respectively. This represents the number of prediction source layers, which is 6 in this case. To accommodate targets with different shapes for predicting regions of interest, OM-SSD sets the aspect ratio of the default bounding box for the k-th layer to be determined by the following formula. :

[0080] (2).

[0081] when At that time, a width and height dimension were added. The default bounding box, i.e., the aspect ratio changes little for small target feature source layers, and large target aspect ratio changes greatly. This represents the ratio of the default bounding box size to the input image size in the (k+1)th layer.

[0082] When the input image size is A total of 1917 default boxes were generated. For ROI detectors that include the outer edge of the idler roller and the edge of the conveyor belt, large default boxes only reduce the target detection speed and contribute almost nothing to improving accuracy; at the same time, due to the occlusion of the conveyor belt, the aspect ratio of the idler roller extension image varies greatly. Therefore, some prediction source layers used to generate default boxes should be removed, and a wider range of aspect ratios should be expanded.

[0083] Because the idler rollers are cylindrical and the occlusion ratio is not fixed, OM-SSD, based on M-SSD, sets the aspect ratio of the default bounding box in L11 to the same six ratios as its prediction source layers (i.e., enriching the aspect ratios of small targets), and removes L14-L21 from the backbone network, using only L11 and L13 as prediction source layers (i.e., removing large target boxes and using only target boxes suitable for the idler roller size as prediction source layers). The model is trained using the standard SSD loss function.

[0084] (3).

[0085] in, This is the default number of matched boxes. It is the matching index between the default box and the actual box. Multi-class prediction confidence and These are the parameters for the predicted bounding box and the ground truth bounding box. Class confidence loss. and outer frame positioning loss These are the main components of the loss function, and their contributions to the loss function are weighted using factors. Balance, and adjust the weighting factors based on experience. Set to 1.

[0086] Phase 2 includes the following three steps: (1) Detect the edge of the conveyor belt using the Hough linear transformation algorithm and the outer edge of the idler using the template matching algorithm; (2) Use the geometric correction algorithm based on homography transformation to geometrically correct the coordinates of the edge of the conveyor belt and the outer edge of the idler, and calculate the length of the outer extension of the idler accordingly; (3) Calculate the deviation based on the length of the outer extension of the idler.

[0087] Based on the Hough linear transform, the steps for detecting the edge of the conveyor belt are as follows:

[0088] (1) Convert the obtained region of interest (ROI) into a grayscale image;

[0089] (2) Use Gaussian filtering to reduce noise in the grayscale image;

[0090] (3) Use the Canny edge detection operator to obtain the edge image in the region of interest (ROI). ;

[0091] (4) Use Hough linear transform to transform the detected edge image The edge points are mapped to the parameter space, and an accumulator is set in the parameter space;

[0092] (5) Set an appropriate accumulator threshold to filter the parameter space and obtain the potential straight line;

[0093] (6) Set an appropriate angle threshold to filter out candidate straight lines that can represent the edge of the conveyor belt from the potential straight lines;

[0094] (7) Obtain the coordinates of the midpoint of the candidate line. The area below the midpoint coordinates is the edge of the conveyor belt.

[0095] The outer edge image of the idler roller in the ROI can be considered as being obtained by sequentially scaling a standard semicircular arc laterally and longitudinally, and then shearing it laterally. Based on this equivalent process, a template matching-based method for detecting the idler roller shaft end area is proposed. First, a template elliptical arc is generated based on prior transformation parameters. Then, the corresponding template elliptical arcs and edge points are traversed to find the parameters with the highest matching degree, thereby determining the position of the idler roller's outer edge.

[0096] The method described in this paper consists of the following five steps:

[0097] (1) Given radius and vertical scaling factor Any point on the reference arc Determined by the following formula:

[0098] (4).

[0099] in, , and These are used to control horizontal and vertical scaling, respectively.

[0100] (2) Given the transverse shear factor any point on the template elliptical arc Determined by the following formula:

[0101] (5).

[0102] Reuse the edge image generated in the conveyor belt edge detection method , with edge point Starting from the point, count the number of edge points along the elliptical arc path of the template. and the number of effective pixels traversal , , , , , The combination, in which , , These are the minimum values ​​of the radius, vertical scaling factor, and horizontal shearing factor, respectively. , , These are their step sizes, , , The matching rate at the edge point is obtained by the following formula, where the number of radius, vertical scaling factor, and horizontal shearing factor are respectively (these are the number of factors). :

[0103] (6).

[0104] As the position of the ROI changes in the image, let the x-coordinate of the left boundary of the ROI's bounding box be... Its ratio to the width of the conveyor belt image is At each altitude, The range of change is ,but Determined by the following formula:

[0105] (7).

[0106] (4) Traversal Repeat step (3) for all edge points and calculate the matching rate for each point. .

[0107] (5) Sort all edge points in descending order of matching rate and output the top edge points with the highest matching rate. One arc parameter.

[0108] The starting point of the outer edge of the idler roller is generally located at In step (4), speeding up the process by only traversing the edge points of the left half of the idler. Typically, the matching rate of the actual idler outer edge is higher than that of similar interferences; therefore, the parameter with the highest matching rate is used to describe the idler outer edge.

[0109] The slope of the straight line along the edge of the conveyor belt is generally small. The midpoint of the detected straight line can be used as approximation points b and c on the edge of the conveyor belt, and the upper vertex of the detected arc can be used as the upper vertices a and d on the outer edge of the idler roller. According to GB / T 10595-2017, when the deviation between the centerline of the conveyor belt and the centerline of the belt conveyor does not exceed ±5% of the belt width, the conveyor belt is considered to be operating normally; when the deviation continues to exceed ±5%, a warning should be given or the machine should be stopped.

[0110] Deviation amount The calculation formula is expressed as follows:

[0111] (8).

[0112] in, Let a be the double derivative. The second derivative of d. To set a threshold.

[0113] The proposed algorithm was validated on a belt conveyor. Overall, as the deviation (DD) decreased from +5% to -5%, the fluctuation in the detection results gradually increased. This is because the inner idler covers a larger area in the image, resulting in more accurate edge detection. As the outer extension of the inner idler decreases, the detection deviation increases. When only one ROI is detected, the calculation results fluctuate. When the conveyor belt edge or the outer edge of the idler is detected incorrectly, the calculation results will deviate from the standard DD. The deviation detection result curves at the same height show that the horizontal shearing of the idler outer edge image has an irregular effect on the deviation results, indirectly verifying the effectiveness of the proposed idler outer edge detection algorithm. The deviation detection error is less than 0.5%, and the detection time is short, meeting the requirements for real-time detection.

[0114] Step 104: Based on the image of the material discharge port in the material transfer data, perform material accumulation detection to obtain the material accumulation detection result.

[0115] In practice, a camera is used to capture images of the material inlet, and a monitoring system is used to detect the accumulation of material in the images of the material inlet.

[0116] An improved ShuffleNet V2 network model is used to process and analyze images of the material discharge port to detect material accumulation. The detection results are then transmitted to a monitoring host via Ethernet for display. The improved ShuffleNet V2 network model replaces standard convolutions with dilated convolutions, increasing the receptive field without increasing computation. An efficient channel attention module (ECA) is added to the basic unit to improve feature extraction capabilities. The improved ShuffleNet V2 network model's material accumulation detection algorithm utilizes a dark channel image dehazing and enhancement algorithm to dehaze and enhance the acquired images, creating a material discharge port image dataset. This dataset is used to train the improved ShuffleNet V2 network model, which is then used for material accumulation detection.

[0117] Hardware and software for a material accumulation detector were designed using a quad-core Jetson Nano development board with a Cortex-A57 architecture. Experimental results show that real-time detection of material accumulation at the discharge port of a belt conveyor can be achieved with an accuracy of 98.34% and an image processing speed of 23 frames per second.

[0118] ShuffleNet, proposed by Megvii, is a lightweight network model suitable for mobile devices. It uses depthwise separable convolution and channel rearrangement to reduce the number of parameters and computational cost. Standard convolution is characterized by a multi-channel kernel sliding back and forth across the multi-channel input, multiplying the pixels in the receptive field with the corresponding pixels of the kernel and summing the results to generate the output. The depthwise separable convolution structure simulation first performs depthwise convolution, using as many kernels as there are input channels, with one kernel corresponding to one channel, followed by 1×1 pointwise convolution.

[0119] ShuffleNet V2 includes: basic unit and downsampling unit structures.

[0120] In the basic unit, the input feed port image is segmented into two parts. One part is an identity mapping, and the other part is first subjected to a 1×1 standard convolution, and then to a depthwise separable convolution. The convolution stride is 1 for both parts, and the number of input and output channels is equal. Then the two parts are concatenated together, and finally the channel rearrangement operation is performed to obtain the feature map.

[0121] In the downsampling unit, the feature map is directly input into two branches. Both branches undergo a 3×3 depthwise convolution with a stride of 2 to reduce the dimensionality of the feature map and decrease computation. The outputs of the two branches are then concatenated, resulting in a sum of channels that is twice the size of the original input. This increased network width enhances the network's feature extraction capabilities.

[0122] The base model is a ShuffleNet V2 1× network model, injecting depthwise convolutions with a stride of 2 into the downsampling units with dilation rates of 2. The base model uses ECA, an efficient channel attention module that compresses spatial features from the input feature map without dimensionality reduction, enhances channel features using global average pooling, and finally combines channel attention signals for channel-wise multiplication to improve feature extraction capabilities. By incorporating the efficient ECA module into both the ShuffleNet V2 base unit and the downsampling unit, performance is optimized while maintaining the model's lightweight design.

[0123] An improved ShuffleNet V2 network model was used to detect material accumulation in the material discharge port image, and the accuracy of the material accumulation detection result was greater than 99.9%, with short processing time, meeting the requirements of real-time detection.

[0124] Step 105: Based on the material image of the conveyor belt in the material transmission data, perform material particle size detection to obtain the material particle size detection result.

[0125] In practice, the material images of the conveyor belt collected by the image sensor will be subjected to texture analysis to determine the accurate texture image. Since the two-dimensional frequency of the texture in the texture image is proportional to the particle size of the material, the accurate particle size of the material can be determined based on the two-dimensional texture frequency corresponding to the texture image, and then the particle size detection result of the material can be generated. The specific steps in step 105 below will be described in detail.

[0126] Step 106: Based on the conveyor belt surface image in the material transfer data, perform longitudinal tear detection to obtain the longitudinal tear detection result.

[0127] In practical implementation, the image sensor can also capture images of the conveyor belt surface, process and analyze these images, detect surface faults such as longitudinal tears, belt misalignment, and surface damage, and issue timely alarms. In particular, it outputs a shutdown control signal when a longitudinal tear fault is detected. It has advantages such as advanced technology, safety and reliability, convenient installation and use, high accuracy, good real-time performance, and intuitive display. It can prevent major longitudinal tears and belt breakage accidents, equipment damage, production stoppages and personnel casualties, reduce material loss and economic losses, ensure the safe and reliable operation of the conveyor belt, improve production efficiency, and has significant social and economic benefits. It can be widely used in coal, mining, ports, power, chemical and other fields, and is particularly suitable for monitoring conveyor belt surface faults in coal mine production.

[0128] The image sensor is a linear CCD (Charge Coupled Device) camera. High-brightness linear composite light sources emit LED light that illuminates the conveyor belt surface, producing diffuse reflection. The intensity of this diffuse reflection is related to the surface characteristics of the conveyor belt. Each linear CCD camera senses the diffuse reflection through line scanning, capturing an image of one row of conveyor belt surface perpendicular to the running direction each time, and transmitting it to the acquisition and processing switch via an Ethernet interface. The acquisition and processing switch uses the multiple rows of conveyor belt surface images received from each linear CCD camera to form an image. It then uses an image stitching algorithm to stitch the images from multiple linear CCD cameras together to form a single conveyor belt surface image. This image is further processed using image processing and correction algorithms before being transmitted to the server via Ethernet. The server uses fast image processing algorithms and fault image feature extraction, recognition, and localization algorithms to process the conveyor belt surface image, analyze and identify surface faults such as longitudinal tearing, misalignment, and surface damage. Upon detecting a fault, it issues a fault alarm or a shutdown control signal.

[0129] Automatically monitors conveyor belt surface faults, stores and displays conveyor belt surface images in real time, and establishes image archives; automatically alarms and outputs shutdown control signals when longitudinal tearing faults of the conveyor belt are detected online; detects conveyor belt misalignment faults, analyzes the degree of misalignment faults, and outputs alarm signals or shutdown control signals; detects and analyzes conveyor belt surface damage faults, performs fault classification and statistics, generates inspection reports, and provides the degree of surface damage faults.

[0130] Step 107: Based on the point cloud data of the cross-section of the conveyor belt material in the material transmission data, perform material flow monitoring to obtain the material flow result.

[0131] In practice, the lidar is installed directly above the conveyor belt. The lidar collects point cloud data of the material cross-section on the conveyor belt, calculates the material flow rate in real time through a material flow rate detection algorithm, and sends the material flow rate results to the monitoring host for display through an Ethernet ring network.

[0132] First, laser point cloud data of the conveyor belt's transverse profile when unloaded is obtained. Then, the data is fitted using a least-squares polynomial curve. Finally, the area formed between the laser rangefinder and the unloaded transverse profile of the conveyor belt is calculated using the fitted data. Then, laser point cloud data of the lateral contour of the accumulated material within that angular range was measured. The data was then subjected to least squares curve fitting, and the area formed between the laser rangefinder and the lateral contour of the accumulated material was calculated using the fitted data. .

[0133] area and area Subtracting the two yields the instantaneous transverse cross-sectional area of ​​the material accumulated on the conveyor belt. Its mathematical expression formula is:

[0134] (9).

[0135] Calculate using the trapezoidal area integral algorithm and The principle is the same; here we only discuss... The calculation principle will be explained. (See figure) Figure 2 yes The schematic diagram of the calculation principle, the calculation method is to first calculate and The area is obtained by subtracting the two areas. Its mathematical formula is:

[0136] (10).

[0137] In equation (10), The principle diagram is calculated using the trapezoidal element area integral method, as shown below. Figure 3 As shown.

[0138] Approximate the red area in the diagram as a right trapezoid, and let the area of ​​the red area be... , The calculation formula is:

[0139] (11).

[0140] In equation (11), n ​​refers to the number of data points in a scanned frame. The calculation formula is:

[0141] (12).

[0142] In equation (12), , These are the distances from the laser rangefinder's scanning center to two adjacent scanning points on the conveyor belt; , They are , The angle between the laser rangefinder and the horizontal direction;

[0143] The sum of the upper and lower bases of the right trapezoid is Gao Wei .

[0144] The calculation principle diagram is as follows Figure 4 As shown, the calculation formula is:

[0145] (13).

[0146] In the formula, , It is the first and the... n The distance between each scanning point and the center of the laser rangefinder scan. , They are , The angle between the laser rangefinder and the horizontal direction.

[0147] Similarly, the trapezoidal area integration algorithm is used to calculate the area formed between the laser rangefinder and the transverse contour of the stockpile material. Through calculation , Once the instantaneous transverse cross-sectional area value is obtained, it can be substituted into the material flow rate calculation formula (for example, the instantaneous transverse cross-sectional area value multiplied by the material flow velocity) to obtain the material flow rate value on the conveyor belt as the material flow rate result.

[0148] Step 108: Based on the operating audio and thermal infrared image of the conveyor belt idler in the material transfer data, perform idler fault fusion monitoring by combining the operating audio and thermal infrared image to obtain the idler fault monitoring result.

[0149] In practice, a sound sensor (a monitoring sensor) and a thermal infrared image sensor (a monitoring sensor) are placed on one side of the belt conveyor and aligned with the idler roller to be monitored. The sound sensor sends the collected operating audio to the monitoring system, and the thermal infrared image sensor collects thermal infrared images and sends them to the monitoring system. In this way, the monitoring system can perform fusion monitoring of idler roller faults by combining the operating audio and thermal infrared images to obtain accurate idler roller fault monitoring results.

[0150] The specific process is as follows:

[0151] (1) Operational audio monitoring: After time-frequency domain noise reduction based on the operational audio, time-frequency domain features are extracted. In the idler roller fault detection model based on dynamic self-attention, multi-frequency dynamic self-attention processing and global dynamic self-attention processing are performed respectively. The results of the two attention processing are spliced ​​and linearly projected to obtain the first decision-level features of the audio. The first decision-level features include: the confidence vector of the first fault type.

[0152] (2) Thermal Infrared Image Monitoring: Image enhancement and noise reduction are performed on the thermal infrared image. Then, the improved MobileNet-SSD ROI detection model is used to extract the region of interest. The key part detection algorithm is then used to detect key parts in the region of interest to determine the key parts. Thermal infrared features are then extracted from the key parts. Based on the extracted thermal infrared features, roller fault monitoring is performed to obtain the second decision-level features of the thermal infrared image. Among them, the second decision-level features include: the temperature rise vector of the key part or the confidence vector of the second fault type.

[0153] (3) Fusion process: The first decision-level features and the second decision-level features are fused to obtain accurate idler roller fault monitoring results.

[0154] The specific process for running audio monitoring as described in (1) above is as follows:

[0155] Operating audio is the most primitive time-frequency domain feature, containing a large amount of broadband noise. This noise can contaminate the spectrum of faulty roller sounds. Therefore, blindly applying band filtering may filter out useful features. Based on the characteristics of mechanical fault sounds and operating noise, weighting can be applied to the frequency axis to preserve or enhance useful frequency band components and attenuate or eliminate irrelevant frequency band components, thereby improving the signal-to-noise ratio of the time-frequency domain features.

[0156] The feature maps extracted and preprocessed in the time-frequency domain are used as input feature maps and fed into the Multi-Frequency Cross-Correlation Dynamic Self-Attention (MF-Cov DSA) module and the Global Dynamic Self-Attention (Global DSA) module, respectively, to obtain multi-frequency cross-correlation feature vectors and global correlation feature vectors. These two vectors are concatenated and linearly projected onto the fault category space to obtain the first decision-level features. MF-Cov DSA and Global Dynamic Self-Attention are the main feature extraction modules of the model, both based on the Transformer's dynamic self-attention module.

[0157] Standard Transformers or visual detection Transformers utilize dynamic self-attention modules to encode and decode the input. Positional encoding is added to the inputs of both the encoder and decoder, and the decoder ultimately outputs the detection result. The dynamic self-attention module uses h sets of learnable projection matrices to map the input to h sets of query / key / value word embeddings to obtain feature maps of different regions of interest. The dimension of each word embedding is reduced to 1 / h. A self-attention operation is performed on each set of reduced-dimensional word embeddings, and the results are concatenated along the dimension axis. The concatenated feature map is used as input, and this process is repeated R times to obtain the module's output, which has the same size as the input.

[0158] The query and key word embeddings for each head use the same projection matrix to reduce the number of parameters, lower the risk of overfitting, and reduce training difficulty. In the low-dimensional feature space, the time series length remains unchanged, while the feature dimension is reduced to its original value. Each element in each dimension is connected to all elements at that time step through the same column of learnable parameters, and elements in the same dimension share the same column of learnable parameters. At this point, each element of the word embedding has a global receptive field at its current time step. Establishing such global connections helps the model perceive common noise, preventing useful weak features from being overwhelmed by strong energy noise. The multi-head self-attention operation dynamically establishes the correlation between the time-frequency domain feature maps on the time axis using the dot product between row vectors in the query / key word embeddings. This correlation is established in the low-dimensional feature space, enhancing the features in the value word embeddings that play a key role in the classification result along the time axis. The query / key word embeddings are dynamically established based on the input features. This dynamic self-attention mechanism allows the model to notice periodic stripes or continuous frequency band energy on the time-frequency domain feature map using a very shallow structure. The shallow structure means that the model has a strong ability to transfer and extract features and is easy to execute in parallel.

[0159] The specific process for thermal infrared image monitoring (2) is as follows:

[0160] (21) Based on Hough linear transformation, detect the edge of the conveyor belt (specifically, the detection process corresponding to the conveyor belt deviation detection above).

[0161] (22) Determine the outer bearing area and shaft end based on the edge of the conveyor belt, and determine the temperature rise value of the outer bearing area. Temperature rise value of shaft end area .

[0162] (14).

[0163] Where 'a' represents different key parts, and 'O' and 'S' represent the external load-bearing area and the shaft end area, respectively. A matrix representing the thermal infrared temperature values ​​of different key components. This means flattening the matrix into a one-dimensional vector. To sort a one-dimensional vector in descending order, for The number of elements.

[0164] Ambient temperature is a physical quantity representing the degree of hotness or coldness of the surrounding environment. While the surrounding environment changes with the scene and the target is not clearly defined, it exhibits significant statistical characteristics. Therefore, extracting ambient temperature is transformed into a statistical problem. Generally, targets with abnormal temperatures account for a small percentage of the scene, and the ambient temperature is the average temperature of most targets in the scene. Based on this prior knowledge, the thermal infrared ambient temperature is defined as the mean of the median portion of the scene's thermal infrared temperature values, expressed as follows:

[0165] (15).

[0166] in, This is a matrix of thermal infrared temperature values. The number of elements in the matrix. Temperature rise in the outer bearing area and shaft end area. and It can be represented as follows:

[0167] (16).

[0168] (23) Determine the temperature rise value of the external bearing area Is it less than or equal to the temperature rise of the external load-bearing area under fault-free conditions? And the shaft end temperature rise value Is it less than or equal to the temperature rise value of the shaft end area under fault-free conditions? If yes, it is considered normal; otherwise, proceed to step (24).

[0169] (24) Determine the relative temperature rise index of the outer bearing area And determine the relative temperature rise index Does it exceed the high threshold of the external load-bearing area temperature rise index? If yes, it is determined to be a simplified wear-through fault (e.g., marked as D2); otherwise, proceed to step (25).

[0170] The temperature rise of critical components differs between fault-free and partial fault conditions. Using the relative change in temperature rise between fault-free and fault-prone conditions as a criterion can more clearly reflect the fault category. The relative temperature rise index is defined as follows:

[0171] (17).

[0172] in, This indicates the temperature rise of critical components (external load-bearing area or shaft end area) under fault-free conditions.

[0173] (25) Determine the relative temperature rise index Is it greater than the low threshold of the external load-bearing area temperature rise index? If so, it is determined to be a card rotation failure (e.g., the marker symbol is D1); otherwise, proceed to step (26).

[0174] (26) Obtain the relative temperature rise index of the shaft end area (using the above calculations) The formula is obtained to determine the relative temperature rise index of the shaft end region. Is it greater than the threshold value for temperature rise in the shaft end area? If yes, it is determined to be a bearing corrosion failure (e.g., marked with the symbol B2); otherwise, it is determined to be another failure.

[0175] The corresponding second-level decision features include: normal results, simple wear-through faults, jamming faults, bearing corrosion faults or other faults, and their respective confidence vectors.

[0176] The above scheme enables the use of monitoring sensors to detect material transport data from belt conveyors. This multi-dimensional material transport data allows for foreign object identification processing of the material transport video, yielding accurate foreign object identification results; conveyor belt misalignment detection of the conveyor belt operation images, yielding accurate conveyor belt misalignment detection results; material accumulation detection of the discharge port images, yielding accurate material accumulation detection results; material particle size detection of the conveyor belt material images, yielding accurate material particle size detection results; longitudinal tear detection of the conveyor belt surface images, yielding accurate longitudinal tear detection results; material flow monitoring of the conveyor belt material cross-sectional point cloud data, yielding accurate material flow rate results; and the fusion monitoring of the conveyor belt idler's operating audio and thermal infrared images, yielding idler fault monitoring results. This allows for multi-faceted detection of the belt conveyor, including foreign object identification, conveyor belt misalignment, material accumulation, material particle size, longitudinal tearing, material flow rate, and idler roller malfunctions. This ensures that the belt conveyor can detect faults or anomalies in a timely manner through multi-faceted detection, and thus take timely action to ensure the normal operation of the belt conveyor.

[0177] In some embodiments, step 105 includes:

[0178] Step 1051: Obtain the predetermined segmentation threshold.

[0179] In some embodiments, the process of determining the segmentation threshold includes:

[0180] Step A1: Determine the standard material image for the conveyor belt (selected based on experience) and use the standard material image as a reference image.

[0181] Step A2 involves normalizing the pixels in the reference image to obtain a normalized reference image. The pixel values ​​in the normalized reference image are values ​​within the range [0,1].

[0182] Step A3: Perform median filtering on the normalized reference image to obtain a median-filtered reference image.

[0183] In practice, the purpose of median filtering is to reduce the noise in the normalized reference image, and the noise removed is some pixels with small fluctuations during acquisition.

[0184] Step A4: Perform multi-channel filtering on the median-filtered reference image (e.g., using a Gabor filter bank for multi-channel filtering) to obtain the filtered reference image.

[0185] Step A5: The mean value is obtained by averaging each pixel in the filtered reference image.

[0186] In practice, the mean is calculated. The formula is:

[0187] (18).

[0188] in, Let be the filtered reference image, m be the number of different scales in the multi-channel filtering, n be the number of different directions, P be the number of pixel rows in the filtered reference image, and Q be the number of pixel columns in the filtered reference image.

[0189] Step A6: Based on the mean, perform standard deviation processing on each pixel in the filtered reference image to obtain the standard deviation.

[0190] In practical implementation, standard deviation The calculation formula is:

[0191] (19).

[0192] Step A7: Determine the difference between each pixel in the filtered reference image and the mean value to obtain an initial feature difference map.

[0193] In practice, the corresponding initial feature difference map for:

[0194] (20).

[0195] This initial feature difference map can characterize the differences between the filtered reference image and the original reference image.

[0196] Step A8: Compare each pixel in the initial feature difference image with the weighted standard deviation, and filter out the pixels in the initial feature difference image that are greater than the weighted standard deviation to obtain the basic feature difference image.

[0197] In practical implementation, the basic feature difference image The formula for determining it is:

[0198] (twenty one).

[0199] in, Standard deviation The weight value.

[0200] Step A9: Fuse the pixels of different scale channels in the same direction in the basic feature difference image to obtain the first basic fusion result.

[0201] In practical implementation, in multi-channel texture analysis of basic feature difference images, since the information of different channels is usually incomplete and uncertain, fusing feature images of each frequency band can reduce the false alarm rate.

[0202] Therefore, we first fuse pixels from channels of different scales in the same direction to obtain the first basic fusion result. The formula is:

[0203] (22), where S is the number of scale channels.

[0204] Step A10: Fuse the pixels in the first basic fusion result in adjacent directions to obtain a basic fused image.

[0205] Because texture similarity generally occurs in at least two adjacent directions, if it only exists in the same direction, it is very likely that the textures are not the same. Therefore, the second step is to fuse the textures in adjacent directions to obtain a basic fused image. The formula is:

[0206] (twenty three).

[0207] in, and K represents the first basic fusion result of two adjacent directions, where K is the number of directions.

[0208] Step A11: The maximum value of the pixels in the base fused image is used as the segmentation threshold.

[0209] In practice, the segmentation threshold Th is: (twenty four).

[0210] The above method can obtain an accurate segmentation threshold, which facilitates subsequent segmentation processing.

[0211] Step 1052: Extract the conveyor belt material image from the material transport data.

[0212] Step 1053: Filter the image of the conveyor belt material to obtain a filtered feature image.

[0213] In some embodiments, step 1053 includes:

[0214] Step 10531: Normalize the pixels in the conveyor belt material image to obtain a normalized material image.

[0215] Step 10532: Perform median filtering on the normalized conveyor belt material image to obtain a median-filtered material image.

[0216] Step 10533: Perform multi-channel filtering (e.g., using a Gabor filter bank) on the median-filtered material image to obtain the filtered feature image. .

[0217] The above method can yield accurate filtered feature images.

[0218] Step 1054: Determine the feature difference image corresponding to the filtered feature image.

[0219] In some embodiments, determining the feature difference image corresponding to the filtered feature image includes:

[0220] Step 10541: Retrieve the pre-determined mean. and the predetermined standard deviation .

[0221] Step 10542: Determine the difference between each pixel in the filtered feature image and the mean value to obtain the feature difference map to be determined. .

[0222] The specific formula is as follows: (25).

[0223] Step 10543: Compare each pixel in the undetermined feature difference image with the weighted standard deviation, and filter out the pixels in the undetermined feature difference image that are greater than the weighted standard deviation to obtain the feature difference image. .

[0224] The specific formula is as follows: (26).

[0225] The above method can yield accurate feature difference images.

[0226] Step 1055: Perform feature fusion based on the feature difference image to obtain a fused image.

[0227] In some embodiments, step 1055 includes:

[0228] Step 10551: Fuse the pixels of different scale channels in the same direction in the feature difference image to obtain the first fusion result.

[0229] The specific formula is as follows:

[0230] (27). Where S is the number of scale channels.

[0231] Step 10552: Merge the pixels in the first fusion result in adjacent directions to obtain the fused image.

[0232] (28).

[0233] in, and For the two first fusion results in adjacent directions, K is the number of directions.

[0234] The above method can yield an accurate fused image.

[0235] Step 1056: Use the segmentation threshold to segment the fused image to extract the material texture portion of the fused image and obtain a material texture image.

[0236] In some embodiments, step 1056 includes:

[0237] Step 10561: Compare the pixels in the fused image with the segmentation threshold.

[0238] Step 10561: Set the gray values ​​of pixels in the fused image that are greater than or equal to the segmentation threshold to the highest gray value (e.g., 1), and set the gray values ​​of pixels in the fused image that are less than the segmentation threshold to the lowest gray value (e.g., 0), to obtain the material texture image. .

[0239] The specific formula is as follows: (29).

[0240] The above method can obtain accurate material texture images.

[0241] Step 1057: Determine the texture period based on the material texture image, and determine the material particle size detection result based on the texture period.

[0242] In some embodiments, step 1057 includes:

[0243] Step 10571: Perform Fast Fourier Transform (FFT) processing on the material texture image to obtain two-dimensional frequency domain data.

[0244] Step 10572: Determine the texture period based on the two-dimensional frequency domain data, and determine the material particle size corresponding to the texture period.

[0245] In practice, the particle size of the material corresponding to each texture period is stored in advance (e.g., table storage, key-value pair storage, graphic storage, or formula storage). After obtaining the texture period, the corresponding particle size can be directly retrieved (or calculated according to a predetermined ratio).

[0246] Step 10573: Use the particle size of the material as the particle size detection result.

[0247] The above solution ensures accurate particle size detection results, which can then be displayed so that users can promptly obtain information about the particle size of the material.

[0248] It should be noted that the method in this embodiment can be executed by a single device, such as a computer or server. The method can also be applied in a distributed scenario, where multiple devices cooperate to complete the task. In such a distributed scenario, one of these devices may execute only one or more steps of the method in this embodiment, and the multiple devices will interact with each other to complete the method described.

[0249] It should be noted that the above description describes some embodiments of this application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in a different order than that shown in the above embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0250] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides an intelligent monitoring and detection system for belt conveyors.

[0251] refer to Figure 5 The system includes:

[0252] The data detection module 201 is configured to detect the material transfer data of the belt conveyor using monitoring sensors;

[0253] The foreign object identification module 202 is configured to perform foreign object identification processing based on the material conveying video in the material transmission data, and determine the foreign object identification processing result;

[0254] The belt misalignment detection module 203 is configured to perform conveyor belt misalignment detection based on the conveyor belt running image in the material transmission data, and obtain the conveyor belt misalignment detection result;

[0255] The material accumulation detection module 204 is configured to perform material accumulation detection based on the material discharge port image in the material transmission data, and obtain the material accumulation detection result.

[0256] The particle size detection module 205 is configured to perform particle size detection based on the conveyor belt material image in the material transmission data, and obtain the particle size detection result.

[0257] The longitudinal tear detection module 206 is configured to perform longitudinal tear detection based on the conveyor belt surface image in the material transfer data, and obtain the longitudinal tear detection result;

[0258] The material flow detection module 207 is configured to monitor the material flow based on the point cloud data of the cross-section of the conveyor belt material in the material transmission data, and obtain the material flow result.

[0259] The idler roller fault detection module 208 is configured to perform idler roller fault fusion monitoring based on the operating audio and thermal infrared images of the conveyor belt idler rollers in the material transfer data, and obtain the idler roller fault monitoring results.

[0260] In some embodiments, the particle size detection module 205 is specifically configured as follows:

[0261] Obtain a predetermined segmentation threshold;

[0262] Extract the conveyor belt material image from the material transport data;

[0263] The material image of the conveyor belt is filtered to obtain a filtered feature image;

[0264] Determine the feature difference image corresponding to the filtered feature image;

[0265] Based on the feature difference image, feature fusion is performed to obtain a fused image;

[0266] The fused image is segmented using the segmentation threshold to extract the material texture portion of the fused image, thereby obtaining a material texture image.

[0267] The texture period is determined based on the material texture image, and the material particle size detection result is determined based on the texture period.

[0268] In some embodiments, the particle size detection module 205 is further configured to:

[0269] Determine a standard material image for the conveyor belt and use the standard material image as a reference image;

[0270] The pixels in the reference image are normalized to obtain a normalized reference image;

[0271] The normalized reference image is subjected to median filtering to obtain a median-filtered reference image.

[0272] The median-filtered reference image is subjected to multi-channel filtering to obtain a filtered reference image.

[0273] The mean value is obtained by averaging each pixel in the filtered reference image.

[0274] The standard deviation is obtained by performing standard deviation processing on each pixel in the filtered reference image based on the mean.

[0275] The difference between each pixel in the filtered reference image and the mean is determined to obtain an initial feature difference map;

[0276] Each pixel in the initial feature difference image is compared with the weighted standard deviation. Pixels in the initial feature difference image that are greater than the weighted standard deviation are selected to obtain the basic feature difference image.

[0277] The pixels of different scale channels in the same direction in the basic feature difference image are fused to obtain the first basic fusion result;

[0278] The pixels in the first basic fusion result are fused in adjacent directions to obtain a basic fused image;

[0279] The maximum value of the pixels in the base fused image is used as the segmentation threshold.

[0280] In some embodiments, the particle size detection module 205 is further configured to:

[0281] The pixels in the conveyor belt material image are normalized to obtain a normalized material image;

[0282] The normalized conveyor belt material image is subjected to median filtering to obtain a median-filtered material image.

[0283] The median-filtered material image is subjected to multi-channel filtering to obtain the filtered feature image.

[0284] In some embodiments, the particle size detection module 205 is further configured to:

[0285] Retrieve the predetermined mean and predetermined standard deviation;

[0286] The difference between each pixel in the filtered feature image and the mean is determined to obtain the feature difference map to be determined;

[0287] Each pixel in the undetermined feature difference image is compared with the weighted standard deviation, and the pixels in the undetermined feature difference image that are greater than the weighted standard deviation are selected to obtain the feature difference image.

[0288] In some embodiments, the particle size detection module 205 is further configured to:

[0289] The pixels of different scale channels in the same direction in the feature difference image are fused to obtain the first fusion result;

[0290] The pixels in the first fusion result are fused in adjacent directions to obtain the fused image.

[0291] In some embodiments, the particle size detection module 205 is further configured to:

[0292] The pixels in the fused image are compared with the segmentation threshold;

[0293] The gray values ​​of pixels in the fused image that are greater than or equal to the segmentation threshold are set to the highest gray value, and the gray values ​​of pixels in the fused image that are less than the segmentation threshold are set to the lowest gray value, thus obtaining the material texture image.

[0294] In some embodiments, the particle size detection module 205 is further configured to:

[0295] The material texture image is processed by Fast Fourier Transform to obtain two-dimensional frequency domain data;

[0296] The texture period is determined based on the two-dimensional frequency domain data, and the particle size of the material corresponding to the texture period is determined.

[0297] The particle size of the material is used as the particle size detection result.

[0298] For ease of description, the above system is described by dividing it into various modules based on their functions. Of course, in implementing this application, the functions of each module can be implemented in one or more software and / or hardware.

[0299] The system described in the above embodiments is used to implement the corresponding method in any of the foregoing embodiments and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0300] Based on the same inventive concept, corresponding to any of the above embodiments, this application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any of the above embodiments.

[0301] Figure 6 This embodiment illustrates a more specific hardware structure of an electronic device. The device may include a processor 1010, a memory 1020, an input / output interface 1030, a communication interface 1040, and a bus 1050. The processor 1010, memory 1020, input / output interface 1030, and communication interface 1040 are interconnected internally via the bus 1050.

[0302] The processor 1010 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this specification.

[0303] The memory 1020 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 1020 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 1020 and is called and executed by the processor 1010.

[0304] The input / output interface 1030 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components within the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touchscreens, microphones, various sensors, etc., while output devices may include displays, speakers, vibrators, indicator lights, etc.

[0305] The communication interface 1040 is used to connect a communication module (not shown in the figure) to enable communication between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0306] Bus 1050 includes a pathway for transmitting information between various components of the device, such as processor 1010, memory 1020, input / output interface 1030, and communication interface 1040.

[0307] It should be noted that although the above-described device only shows the processor 1010, memory 1020, input / output interface 1030, communication interface 1040, and bus 1050, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0308] The electronic devices described above are used to implement the corresponding methods in any of the foregoing embodiments and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0309] Based on the same inventive concept, corresponding to the methods of any of the above embodiments, this application also provides a non-transitory computer-readable storage medium that stores computer instructions for causing the computer to perform the methods described in any of the above embodiments.

[0310] The computer-readable medium of this embodiment includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0311] The computer instructions stored in the storage medium of the above embodiments are used to cause the computer to perform the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0312] Based on the same concept, corresponding to any of the above embodiments, this application also provides a computer program product, including computer program instructions, which, when run on a computer, cause the computer to perform the method described in any of the above embodiments, and have the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0313] It is understood that before using the technical solutions of the various embodiments in this application, users will be informed of the type, scope of use, and usage scenarios of the personal information involved in an appropriate manner, and user authorization will be obtained.

[0314] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose, based on the prompt message, whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media performing the operations described in this application.

[0315] As an optional but not limited implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0316] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this application. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this application.

[0317] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of the embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0318] Additionally, to simplify the description and discussion, and to avoid obscuring the embodiments of this application, the well-known power / ground connections to integrated circuit (IC) chips and other components may or may not be shown in the provided drawings. Furthermore, systems may be illustrated in block diagram form to avoid obscuring the embodiments of this application, and this also takes into account the fact that the details of implementation of these block diagram systems are highly dependent on the platform on which the embodiments of this application will be implemented (i.e., these details should be fully understood by those skilled in the art). While specific details (e.g., circuits) have been set forth to describe exemplary embodiments of this application, it will be apparent to those skilled in the art that the embodiments of this application can be implemented without these specific details or with variations thereof. Therefore, these descriptions should be considered illustrative rather than restrictive.

[0319] Although this application has been described in conjunction with specific embodiments thereof, many substitutions, modifications, and variations of these embodiments will be apparent to those skilled in the art from the foregoing description. For example, other memory architectures (e.g., dynamic RAM (DRAM)) may be used with the embodiments discussed.

[0320] The embodiments of this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the claims of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the embodiments of this application should be included within the protection scope of this application.

Claims

1. A method for intelligent monitoring and detection of a belt conveyor, characterized in that, include: Utilize monitoring sensors to detect material transfer data of belt conveyors; Based on the material conveying video in the material transmission data, foreign object identification processing is performed, and the result of foreign object identification processing is determined. Based on the conveyor belt running image in the material transfer data, conveyor belt misalignment detection is performed to obtain the conveyor belt misalignment detection result; Based on the image of the material discharge port in the material transfer data, material accumulation detection is performed to obtain the material accumulation detection result; Based on the material image of the conveyor belt in the material transmission data, the particle size of the material is detected to obtain the particle size detection result. Based on the conveyor belt surface image in the material transfer data, longitudinal tear detection is performed to obtain the longitudinal tear detection result; Based on the point cloud data of the material cross-section of the conveyor belt in the material transfer data, material flow rate is monitored to obtain the material flow rate result; Based on the operating audio and thermal infrared images of the conveyor belt idlers in the material transfer data, the operating audio and thermal infrared images are fused together for idler fault monitoring to obtain idler fault monitoring results. The step of performing material particle size detection based on the conveyor belt material image in the material transport data to obtain the material particle size detection result includes: Obtain a predetermined segmentation threshold; Extract the conveyor belt material image from the material transport data; The material image of the conveyor belt is filtered to obtain a filtered feature image; Determine the feature difference image corresponding to the filtered feature image; Based on the feature difference image, feature fusion is performed to obtain a fused image; The fused image is segmented using the segmentation threshold to extract the material texture portion of the fused image, thereby obtaining a material texture image. The texture period is determined based on the material texture image, and the material particle size detection result is determined based on the texture period.

2. The method according to claim 1, characterized in that, The process of determining the segmentation threshold includes: Determine a standard material image for the conveyor belt and use the standard material image as a reference image; The pixels in the reference image are normalized to obtain a normalized reference image; The normalized reference image is subjected to median filtering to obtain a median-filtered reference image. The median-filtered reference image is subjected to multi-channel filtering to obtain a filtered reference image. The mean value is obtained by averaging each pixel in the filtered reference image. The standard deviation is obtained by performing standard deviation processing on each pixel in the filtered reference image based on the mean. The difference between each pixel in the filtered reference image and the mean is determined to obtain an initial feature difference map; Each pixel in the initial feature difference image is compared with the weighted standard deviation. Pixels in the initial feature difference image that are greater than the weighted standard deviation are selected to obtain the basic feature difference image. The pixels of different scale channels in the same direction in the basic feature difference image are fused to obtain the first basic fusion result; The pixels in the first basic fusion result are fused in adjacent directions to obtain a basic fused image; The maximum value of the pixels in the base fused image is used as the segmentation threshold.

3. The method according to claim 1, characterized in that, The step of filtering the conveyor belt material image to obtain a filtered feature image includes: The pixels in the conveyor belt material image are normalized to obtain a normalized material image; The normalized conveyor belt material image is subjected to median filtering to obtain a median-filtered material image. The median-filtered material image is subjected to multi-channel filtering to obtain the filtered feature image.

4. The method according to claim 1, characterized in that, Determining the feature difference image corresponding to the filtered feature image includes: Retrieve the predetermined mean and predetermined standard deviation; The difference between each pixel in the filtered feature image and the mean is determined to obtain the feature difference map to be determined; Each pixel in the undetermined feature difference image is compared with the weighted standard deviation, and the pixels in the undetermined feature difference image that are greater than the weighted standard deviation are selected to obtain the feature difference image.

5. The method according to claim 1, characterized in that, The step of performing feature fusion based on the feature difference image to obtain a fused image includes: The pixels of different scale channels in the same direction in the feature difference image are fused to obtain the first fusion result; The pixels in the first fusion result are fused in adjacent directions to obtain the fused image.

6. The method according to claim 1, characterized in that, The step of segmenting the fused image using the segmentation threshold to extract the material texture portion of the fused image and obtain a material texture image includes: The pixels in the fused image are compared with the segmentation threshold; The gray values ​​of pixels in the fused image that are greater than or equal to the segmentation threshold are set to the highest gray value, and the gray values ​​of pixels in the fused image that are less than the segmentation threshold are set to the lowest gray value, thus obtaining the material texture image.

7. The method according to claim 1, characterized in that, The step of determining the texture period based on the material texture image and determining the material particle size detection result based on the texture period includes: The material texture image is processed by Fast Fourier Transform to obtain two-dimensional frequency domain data; The texture period is determined based on the two-dimensional frequency domain data, and the particle size of the material corresponding to the texture period is determined. The particle size of the material is used as the particle size detection result.

8. An intelligent monitoring and detection system for a belt conveyor, characterized in that, include: The data detection module is configured to detect the material transfer data of the belt conveyor using monitoring sensors; The foreign object identification module is configured to perform foreign object identification processing based on the material conveying video in the material transmission data, and determine the foreign object identification processing result; The belt misalignment detection module is configured to perform conveyor belt misalignment detection based on the conveyor belt running image in the material transmission data, and obtain the conveyor belt misalignment detection result; The material accumulation detection module is configured to perform material accumulation detection based on the material discharge port image in the material transmission data, and obtain the material accumulation detection result. The particle size detection module is configured to perform particle size detection based on the conveyor belt material image in the material transmission data, and obtain the particle size detection result. The longitudinal tear detection module is configured to perform longitudinal tear detection based on the conveyor belt surface image in the material transport data, and obtain the longitudinal tear detection result; The material flow detection module is configured to monitor the material flow based on the point cloud data of the cross-section of the material in the conveyor belt in the material transmission data, and obtain the material flow result. The idler roller fault detection module is configured to perform idler roller fault fusion monitoring based on the operating audio and thermal infrared images of the conveyor belt idler rollers in the material transfer data, and obtain the idler roller fault monitoring results. The particle size detection module is specifically configured as follows: Obtain a predetermined segmentation threshold; Extract the conveyor belt material image from the material transport data; The material image of the conveyor belt is filtered to obtain a filtered feature image; Determine the feature difference image corresponding to the filtered feature image; Based on the feature difference image, feature fusion is performed to obtain a fused image; The fused image is segmented using the segmentation threshold to extract the material texture portion of the fused image, thereby obtaining a material texture image. The texture period is determined based on the material texture image, and the material particle size detection result is determined based on the texture period.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

Images