Vision-based tubular belt conveyor coal flow monitoring and protecting system and method

Abstract

The invention relates to a tubular belt conveyor coal flow monitoring and protecting system and method based on vision, and the system comprises a multi-sensor array module which comprises at least one group of 3D vision sensors and industrial cameras which are arranged along the longitudinal direction of a conveying belt, and is used for collecting the three-dimensional section image of the coal flow in a tubular belt; a coal flow edge detection algorithm is built in the edge calculation unit, and the geometric contour of the coal flow section is extracted in real time by fusing the RGB image and the depth data; the coal quantity dynamic calculation module is used for calculating the instantaneous coal quantity based on the geometric contour of the coal flow cross section and the data of the conveyor belt speed sensor; and the grading early warning module is used for setting a multi-grade coal quantity threshold value, obtaining a sampling period coal quantity according to the corrected instantaneous coal quantity, triggering a sound-light alarm, a variable frequency speed regulation signal or an emergency shutdown instruction according to a comparison value of the sampling period coal quantity and the multi-grade coal quantity threshold value, and performing interlocking control on a conveyor driving device.

Description

Technical Field

[0001] The invention relates to a vision-based tubular belt conveyor coal flow monitoring and protection system and method, belonging to the technical field of coal mine conveying equipment control. Background Art

[0002] As an important equipment for coal transportation, the safe and stable operation of tubular belt conveyors is crucial to coal mine production. With the development of intelligent coal mines, higher requirements are placed on the coal flow monitoring and control of conveying equipment. At present, the coal flow monitoring of tubular belt conveyors mainly adopts technical means such as weighing sensors, radars or single vision systems, but there are still many technical difficulties in practical applications.

[0003] Traditional tubular belt conveyor coal flow monitoring mainly relies on contact sensors, such as weighing rollers, tension sensors, etc. Although this type of sensor has a simple structure, it has problems such as large response delay, difficult installation and maintenance, and susceptibility to environmental influences, making it difficult to meet the needs of high-precision real-time monitoring. The Chinese invention patent with publication number CN118701643A discloses a machine vision-based belt conveyor coal quantity monitoring device and monitoring method. This method uses a line laser to illuminate a heavy-duty conveyor belt to form the contour of the coal and the belt, and then uses a digital camera to capture the red contour reflection line to extract coal combustion characteristic information and calculate the volume of coal powder. However, this technology is mainly designed for open belt conveyors and is difficult to adapt to the special closed structure of tubular belts.

[0004] In recent years, machine vision technology has been widely used in the field of coal flow monitoring. CN113911629A proposed an intelligent coal flow transportation system, which acquires image data and extracts the cross-sectional profile of coal by setting up laser transmitters and cameras, and calculates the coal quantity data in combination with the transmission speed. CN118183225A discloses an intelligent coal flow control system for belt conveyors based on machine vision. The system can determine the real-time transportation flow rate based on the coal flow image and adjust the transportation speed accordingly. Although these technologies have achieved good results on open belt conveyors, for tubular belt conveyors, due to their special closed tubular structure, it is difficult for traditional 2D vision systems to accurately obtain the actual distribution of coal flow in the tubular space, resulting in insufficient accuracy in coal quantity calculation. Summary of the invention

[0005] In order to solve the problems of large response delay, difficult installation and maintenance in traditional tubular belt coal quantity monitoring, and the defect that the existing visual detection scheme cannot accurately calculate the actual filling amount of special-shaped tubular sections and lead to misjudgment, and at the same time, to address the safety hazard of tube expansion accidents that often occur in instantaneous large coal volume impact conditions, the present invention provides a vision-based tubular belt conveyor coal flow monitoring and protection system and method.

[0006] The technical solution of the present invention is as follows:

[0007] On the one hand, the present invention provides a vision-based tubular belt conveyor coal flow monitoring and protection system, comprising:

[0008] Multi-sensor array module: comprising at least one set of 3D vision sensors and industrial cameras arranged longitudinally along the conveyor belt, for synchronously collecting stereoscopic cross-sectional images of the coal flow in the tubular belt;

[0009] Edge computing unit: Built-in coal flow edge detection algorithm, which extracts the geometric contour of the coal flow section in real time by fusing RGB images and depth data;

[0010] Coal quantity dynamic calculation module: Based on the geometric profile of the coal flow section and the conveyor belt speed sensor data, the instantaneous coal quantity is calculated using the integral method, and the material stacking density coefficient is dynamically corrected. The instantaneous coal quantity is corrected according to the material stacking density coefficient;

[0011] Gradual warning module: set multi-level coal quantity thresholds, obtain the sampling period coal quantity according to the corrected instantaneous coal quantity, trigger the sound and light alarm, frequency conversion speed regulation signal or emergency stop command according to the comparison value between the sampling period coal quantity and the multi-level coal quantity thresholds, and interlock control the conveyor drive device.

[0012] As a preferred embodiment, the 3D vision sensor adopts a binocular stereo camera array or a ToF laser scanner, which is installed above the opening side of the tubular belt to cover the entire cross-section of the conveyor belt at an inclination angle of 45-60°.

[0013] As a preferred implementation, the coal flow edge detection algorithm includes:

[0014] Adaptive coal flow region segmentation based on HSV color space;

[0015] Edge refinement extraction combining Canny operator and morphological processing;

[0016] The inner wall baseline of the tubular belt is fitted by the RANSAC algorithm, and the actual filling rate of the coal flow section is calculated to extract the geometric contour of the coal flow section.

[0017] As a preferred implementation, the coal quantity dynamic calculation module integrates tension sensor data, establishes a coal quantity-belt tension coupling model, performs multi-source data fusion through Kalman filtering, and corrects instantaneous coal quantity calculation errors based on multi-source fusion data.

[0018] As a preferred implementation, the emergency stop command triggering condition is that the coal volume exceeds the threshold value for three consecutive sampling cycles and the gradient change rate is greater than 5% / s, and the upstream feeding equipment is simultaneously linked to an emergency stop.

[0019] On the other hand, the present invention also provides a vision-based tubular belt conveyor coal flow monitoring and protection method, comprising the following steps:

[0020] A multi-sensor array module including at least one set of 3D vision sensors and industrial cameras arranged longitudinally along the conveyor belt is provided to synchronously collect stereoscopic cross-sectional images of the coal flow in the tubular belt;

[0021] Through the coal flow edge detection algorithm that integrates RGB images and depth data, the geometric contour of the coal flow cross section is extracted in real time;

[0022] Based on the geometric profile of the coal flow section and the data of the conveyor belt speed sensor, the instantaneous coal quantity is calculated by the integral method, and the material stacking density coefficient is dynamically corrected. The instantaneous coal quantity is corrected according to the material stacking density coefficient;

[0023] Set multi-level coal quantity thresholds, obtain the sampling period coal quantity based on the corrected instantaneous coal quantity, trigger the sound and light alarm, frequency conversion speed regulation signal or emergency stop command based on the comparison value between the sampling period coal quantity and the multi-level coal quantity thresholds, and interlock control the conveyor drive device.

[0024] As a preferred embodiment, the 3D vision sensor adopts a binocular stereo camera array or a ToF laser scanner, which is installed above the opening side of the tubular belt to cover the entire cross-section of the conveyor belt at an inclination angle of 45-60°.

[0025] As a preferred implementation, adaptive coal flow region segmentation based on HSV color space;

[0026] Edge refinement extraction combining Canny operator and morphological processing;

[0027] The inner wall baseline of the tubular belt is fitted by the RANSAC algorithm, and the actual filling rate of the coal flow section is calculated to extract the geometric contour of the coal flow section.

[0028] As a preferred implementation, in the step of correcting the instantaneous coal quantity, the tension sensor data is also integrated, a coal quantity-belt tension coupling model is established, multi-source data fusion is performed through Kalman filtering, and the instantaneous coal quantity calculation error is corrected based on the multi-source fusion data.

[0029] As a preferred implementation, the emergency stop command triggering condition is that the coal volume exceeds the threshold value for three consecutive sampling cycles and the gradient change rate is greater than 5% / s, and the upstream feeding equipment is simultaneously linked to an emergency stop.

[0030] On the other hand, the present invention also proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vision-based tubular belt conveyor coal flow monitoring and protection method as described in any embodiment of the present invention is implemented.

[0031] On the other hand, the present invention further proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the vision-based tubular belt conveyor coal flow monitoring and protection method as described in any embodiment of the present invention.

[0032] The beneficial effects of the present invention are:

[0033] Compared with traditional contact sensors (such as weighing rollers), the present invention adopts non-contact ToF sensors and RGB image fusion technology to solve the problems of large response delay and difficult installation and maintenance; through three-dimensional visual reconstruction technology, the defect of existing 2D image detection schemes that cannot accurately calculate the actual filling amount of special-shaped tubular sections is overcome, avoiding misjudgment; through dynamic density compensation and FPGA hardware acceleration, high-precision real-time monitoring is achieved, and the full-link delay is less than 200ms, effectively preventing the occurrence of pipe expansion accidents. Experimental data show that under different coal types and conveying speed conditions, the coal quantity calculation accuracy of this system can reach ±3%, and the response time is stable within 150ms. Compared with traditional systems, it improves monitoring accuracy and shortens response time, significantly reducing the incidence of pipe expansion accidents.

[0034] Additional aspects and advantages of the present invention will be set forth in the following description, and some of them will be apparent from the description, or may be understood by practicing the present invention. In addition, the various aspects and advantages of the present invention may be realized and obtained by the method steps and combinations particularly pointed out in the appended claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a schematic diagram of the system framework of Embodiment 1 of the present invention;

[0036] Figure 2 This is a schematic diagram of the method flow of Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] It should be understood that the step numbers used in this document are only for convenience of description and are not intended to limit the order in which the steps are executed.

[0039] It should be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0040] The terms “include” and “comprising” indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0041] The term "and / or" means and includes any and all possible combinations of one or more of the associated listed items.

[0042] Embodiment 1:

[0043] See also Figure 1 This embodiment provides a vision-based tubular belt conveyor coal flow monitoring and protection system, including: a multi-sensor array module, an edge computing unit, a coal quantity dynamic computing module and a hierarchical warning module, wherein:

[0044] The multi-sensor array module includes at least one group of 3D vision sensors and industrial cameras arranged longitudinally along the conveyor belt. In this embodiment, the multi-sensor array module uses three groups of sensor units, each group of sensor units includes a 3D vision sensor and an industrial camera, and the three groups of sensor units are arranged at equal intervals along the running direction of the tubular belt conveyor, with a spacing of 5 meters. The 3D vision sensor uses a binocular stereo camera with a resolution of 1920×1080 pixels, a frame rate of 30fps, a field of view of 85°, and a working distance range of 0.5-8 meters. The industrial camera uses a global shutter CMOS sensor with a resolution of 2448×2048 pixels, a frame rate of 60fps, a 12mm fixed-focus lens, an IP67 protection level, and an operating temperature range of -20℃ to 70℃.

[0045] The 3D vision sensor is installed above the open side of the tubular belt, covering the entire cross-section of the conveyor belt at an angle of 55°. The sensor is installed at a height of 1.5 times the width of the conveyor belt to ensure that the field of view can fully cover the entire cross-section of the tubular belt. The industrial camera and the 3D vision sensor are installed side by side, with the angle between the optical axes of the two being 5° to ensure that the overlapping area of ​​the field of view is maximized. The multi-sensor array module is fixed to the conveyor frame by a sturdy bracket made of carbon steel with an anti-corrosion surface treatment to withstand vibration and dust in an industrial environment.

[0046] Each sensor unit in the multi-sensor array module is equipped with an independent LED fill light with a luminous flux of 3000 lumens and a color temperature of 5500K, ensuring that clear images can be obtained in low-light environments. The fill light is triggered synchronously with the sensor and only lights up during image acquisition to extend the service life and reduce energy consumption. The sensor unit is also equipped with a dust-proof air curtain system that forms an airflow barrier through compressed air to prevent coal dust from adhering to the surface of the optical lens.

[0047] The multi-sensor array module is connected to the edge computing unit via industrial-grade Gigabit Ethernet and uses TCP / IP protocol for communication, with a data transmission delay of less than 10 milliseconds. The data collected by the sensor includes RGB images and depth images. The RGB image is used for coal flow surface feature recognition, and the depth image is used for coal flow cross-sectional profile extraction. The sampling frequency of the multi-sensor array module can be dynamically adjusted according to the conveyor belt speed. The faster the conveyor belt speed, the higher the sampling frequency, ensuring the continuity and accuracy of data collection.

[0048] The edge computing unit uses an industrial-grade embedded computing platform, equipped with an eight-core processor, a main frequency of 3.0GHz, 16GB RAM and 512GB SSD storage, and runs a real-time Linux operating system. The edge computing unit has a built-in coal flow edge detection algorithm, which extracts the geometric contour of the coal flow section in real time by fusing RGB images and depth data. The edge computing unit adopts a modular design, including an image preprocessing module, a feature extraction module, a contour recognition module, and a data fusion module.

[0049] The image preprocessing module performs denoising, correction and enhancement on the collected RGB images and depth images. Denoising uses a combination of Gaussian filtering and median filtering to effectively suppress random noise and salt and pepper noise in the image. Correction includes geometric correction and photometric correction. Geometric correction eliminates lens distortion, and photometric correction balances image brightness and contrast. Enhancement uses an adaptive histogram equalization algorithm to improve the contrast and detail expression of the image.

[0050] The feature extraction module performs adaptive coal flow area segmentation based on the HSV color space. First, the RGB image is converted to the HSV color space, and then the initial threshold range is set according to the hue (H), saturation (S) and brightness (V) characteristics of the coal: H value is [0,30], S value is [0,0.3], and V value is [0,0.5]. The system automatically adjusts these thresholds by analyzing the image histogram to adapt to different types of coal and lighting conditions. The segmented binary image is optimized through morphological operations (opening and closing operations) to eliminate isolated noise points and fill small holes.

[0051] The contour recognition module combines the Canny operator with morphological processing to extract edges in a refined manner. The Canny operator parameters are set as follows: the high threshold is 30% of the maximum value of the image gradient, the low threshold is 40% of the high threshold, and the Gaussian filter standard deviation is 1.4. After edge detection, the morphological thinning algorithm is applied to extract edge lines with a single pixel width, and then the polygonal approximation algorithm is used to simplify the edge contour, reduce redundant points, and improve the efficiency of subsequent processing.

[0052] The data fusion module fits the inner wall baseline of the tubular belt through the RANSAC algorithm and calculates the actual filling rate of the coal flow section. The number of iterations of the RANSAC algorithm is set to 1000, the inner point threshold is 2 pixels, and the confidence level is 99%. First, the edge points of the inner wall of the tubular belt are identified, and then the geometric model of the tubular belt is fitted to establish a coordinate system. The coal flow contour is projected into the coordinate system, and the ratio of the cross-sectional area of ​​the coal flow to the cross-sectional area of ​​the tubular belt is calculated to obtain the filling rate. The data fusion module also combines the depth information to correct the perspective deformation caused by the viewing angle to improve the accuracy of the filling rate calculation.

[0053] The processing results of the edge computing unit are transmitted to the coal quantity dynamic calculation module via industrial Ethernet. The transmitted data includes information such as the coal flow contour coordinate point set, filling rate, and coal flow height distribution. The data format uses a JSON structure to facilitate subsequent processing and storage. The edge computing unit has a local storage function, which can cache 30 minutes of processing results in the event of a network interruption, and automatically upload after the network is restored to ensure data continuity.

[0054] The coal quantity dynamic calculation module calculates the instantaneous coal quantity based on the geometric profile of the coal flow section and the conveyor belt speed sensor data, and dynamically corrects the material stacking density coefficient. The instantaneous coal quantity is corrected according to the material stacking density coefficient. The coal quantity dynamic calculation module uses an industrial-grade server equipped with a quad-core processor with a main frequency of 2.5GHz, 8GBRAM and 1TBSSD storage, running a database management system and computing engine.

[0055] The coal quantity dynamic calculation module receives data from the conveyor belt speed sensor. The speed sensor uses a non-contact photoelectric encoder with a resolution of 1024 pulses / turn. It is installed at the shaft end of the driven roller of the conveyor belt and calculates the conveyor belt linear speed by measuring the roller speed. The speed data sampling frequency is 100Hz and is transmitted to the coal quantity dynamic calculation module through the CAN bus.

[0056] The coal quantity dynamic calculation module uses the integral method to calculate the instantaneous coal quantity. First, based on the coal flow cross-sectional area S(t) and the conveyor belt speed v(t), calculate the volume of coal passing through the monitoring point per unit time: V(t) = S(t) × v(t) × Δt, where Δt is the sampling time interval. Then, multiply it by the material stacking density coefficient ρ to obtain the instantaneous coal quantity: M(t) = V(t) × ρ. The initial value of the material stacking density coefficient is set to 0.8 tons / cubic meter, and the system will dynamically adjust it according to the actual operating data.

[0057] The coal quantity dynamic calculation module integrates the tension sensor data, establishes the coal quantity-belt tension coupling model, realizes multi-source data fusion through Kalman filtering, and corrects the instantaneous coal quantity calculation error. The tension sensor is installed on the load-bearing section and return section of the conveyor belt. It uses a strain gauge sensor with a range of 0-20kN and an accuracy of ±0.5% of the full scale. The tension data sampling frequency is 50Hz, and it is transmitted to the data acquisition module through a 4-20mA analog signal, and then converted into a digital signal and input into the coal quantity dynamic calculation module.

[0058] The coal quantity-belt tension coupling model is established based on physical principles, considering the relationship between conveyor belt tension and load: T = T 0 +μgL(2m+M), where T is the total tension, T 0 is the initial tension, μ is the friction coefficient, g is the acceleration of gravity, L is the length of the conveyor belt, m is the mass per unit length of the conveyor belt, and M is the mass per unit length of the coal. The coal load is reversely calculated through the real-time measured tension value and compared and integrated with the visual measurement results.

[0059] The Kalman filter algorithm is used to fuse the coal quantity data of visual measurement and tension measurement. The state vector includes the coal quantity and the coal quantity change rate, and the observation vector includes the coal quantity measured by vision and the coal quantity measured by tension. The process noise covariance matrix and the observation noise covariance matrix are obtained through system calibration, and the initial values ​​are set to [[0.01,0],[0,0.001]] and [[0.05,0],[0,0.08]] respectively. The Kalman filter is executed every 100 milliseconds and outputs the fused coal quantity estimate, which is more than 30% more accurate than a single sensor.

[0060] The coal quantity dynamic calculation module also realizes the adaptive adjustment of the material stacking density coefficient. The system analyzes historical data, identifies the coal quantity-tension relationship in the stable operation stage, establishes a regression model, and regularly updates the material stacking density coefficient. When changes in material properties (such as changes in humidity and particle size distribution) are detected, the system automatically adjusts the density coefficient to an adaptive range of 0.7-1.2 tons / cubic meter.

[0061] The graded warning module sets multi-level coal quantity thresholds, obtains the sampling period coal quantity based on the corrected instantaneous coal quantity, triggers the sound and light alarm, frequency conversion speed regulation signal or emergency stop command based on the comparison value between the sampling period coal quantity and the multi-level coal quantity thresholds, and interlocks and controls the conveyor drive device.

[0062] The hierarchical warning module is implemented with a programmable logic controller (PLC) and equipped with redundant power supplies and communication interfaces to ensure system reliability. The hierarchical warning module sets three levels of warning thresholds: the first level warning threshold is 80% of the designed coal volume, the second level warning threshold is 90% of the designed coal volume, and the third level warning threshold is 95% of the designed coal volume.

[0063] When the coal volume reaches the first-level warning threshold, the system triggers an audible and visual alarm, including flashing warning lights on site and sound prompts in the control room, reminding operators to pay attention to the coal flow status. The audible and visual alarm uses a red and yellow LED warning light with a brightness of 2000cd and a visible distance of 50 meters. The volume of the sound alarm is 95dB, the frequency is 2900Hz, and it uses an intermittent alarm mode.

[0064] When the coal quantity reaches the secondary warning threshold, the system generates a variable frequency speed regulation signal, which is connected to the conveyor inverter through the 4-20mA analog output interface to reduce the conveyor belt speed. The speed reduction is 20% of the current speed, and the minimum is not less than 50% of the rated speed to avoid motor overload and startup difficulties. At the same time, the system sends a reduction signal to the upstream feeding equipment, and communicates through the Modbus TCP protocol to request the upstream equipment to reduce the feeding amount.

[0065] When the coal quantity reaches the third-level warning threshold and the coal quantity exceeds the threshold for three consecutive sampling cycles and the gradient change rate is greater than 5% / s, the system triggers an emergency stop command and links the upstream feeding equipment to an emergency stop. The emergency stop command is hard-wired to the safety circuit of the conveyor drive device to cut off the drive power supply. At the same time, an emergency stop command is sent to the upstream feeding equipment through the field bus to achieve linkage control. After the emergency stop, the system records the operating data of the 10 minutes before the stop, including information such as the coal quantity change trend, conveyor belt speed, tension change, etc., to facilitate subsequent analysis of the cause of the fault.

[0066] The hierarchical warning module has a self-diagnosis function and regularly detects the integrity of the sensor status, communication link and control loop. When a system fault is detected, a fault alarm message is generated and corresponding measures are taken according to the fault level, such as downgraded operation or safe shutdown. System faults are divided into three levels: Level 1 faults are non-critical sensor faults, and the system can continue to operate; Level 2 faults are communication interruptions or partial loss of function, and the system is downgraded; Level 3 faults are failures of key components and the system is safely shut down.

[0067] The system is also equipped with a remote monitoring interface that supports integration with upper-level production management systems via industrial Ethernet. The remote monitoring interface uses the OPC UA protocol to provide real-time data, historical trends, alarm information, system status and other information. Authorized users can remotely view the system operation status, download historical data reports, and adjust system parameters through the Web interface. Remote access uses TLS encryption and two-factor authentication to ensure system security.

[0068] Embodiment 2:

[0069] Based on the first embodiment, the 3D vision sensor in this embodiment adopts a ToF laser scanner instead of a binocular stereo camera. The working principle of the ToF laser scanner is based on time-of-flight ranging, emitting laser pulses and measuring the round-trip time of the light signal to calculate the distance. The ToF laser scanner has a resolution of 640×480 pixels, a frame rate of 45fps, a field of view of 70°, a ranging range of 0.4-10 meters, and a ranging accuracy of ±1cm. The ToF laser scanner is also installed above the opening side of the tubular belt, covering the entire cross-section of the conveyor belt at an inclination of 50°.

[0070] The ToF laser scanner has a higher ability to resist environmental interference and can still obtain stable depth information in an environment with high dust concentration. The ToF laser scanner uses a 940nm wavelength laser light source with a power of 1W, which meets the Class 1 laser safety standard and is harmless to the human eye. The ToF laser scanner is equipped with a self-cleaning system, including a compressed air nozzle and a dust cover, which automatically removes dust from the lens surface regularly.

[0071] Since the ToF laser scanner directly outputs the depth image, the image processing process of the edge computing unit is simplified. The system directly extracts the coal flow contour from the depth image without the need for RGB to depth registration, reducing the amount of calculation and the source of error. The data of the ToF laser scanner is transmitted to the edge computing unit via the USB 3.0 interface, and the data transmission rate can reach 5Gbps, meeting the real-time processing requirements.

[0072] Embodiment three:

[0073] On the basis of Example 1, the coal flow edge detection algorithm in this embodiment adopts a deep learning method to replace the traditional image processing method. The deep learning model adopts a convolutional neural network with a U-Net architecture, which is specifically used for image segmentation tasks. The U-Net model consists of two parts: an encoder and a decoder. The encoder consists of 5 convolution blocks, each of which contains two 3×3 convolution layers and a maximum pooling layer; the decoder consists of 4 upsampling blocks and convolution blocks, and the encoder features are fused through jump connections.

[0074] The model input is a 512×512 pixel RGB image and the corresponding depth image, and the output is a binary segmentation mask of the coal flow area. The model is trained using 8,000 annotated images, of which 6,000 are used for training, 1,000 for validation, and 1,000 for testing. The training uses the Adam optimizer, with an initial learning rate of 0.001, which decays to the original 0.1 every 50 epochs, and a total of 200 epochs. The loss function uses the weighted sum of Dice loss and cross entropy loss, with a weight ratio of 1:1.

[0075] The trained model is deployed on the edge computing unit and accelerated using TensorRT, with an inference speed of 25fps. The segmentation accuracy (IoU) of the model on the test set reaches 94.5%, 15 percentage points higher than the traditional method. The deep learning model has stronger environmental adaptability and can handle coal flow segmentation tasks under different lighting conditions, different types of coal and complex backgrounds.

[0076] After the model is deployed, the system automatically collects new image data every 24 hours, and after manual screening and annotation, it is used for incremental learning of the model to continuously improve the model performance. Incremental learning uses a smaller learning rate (0.0001) and fewer training rounds (20 epochs) to retain the model's ability to remember the learned samples while adapting to the new data distribution.

[0077] Embodiment 4:

[0078] This embodiment provides a visual-based method for monitoring and protecting coal flow in a tubular belt conveyor, comprising the following steps:

[0079] S100, setting a multi-sensor array module including at least one group of 3D vision sensors and industrial cameras arranged longitudinally along the conveyor belt, and synchronously collecting a three-dimensional cross-sectional image of the coal flow in the tubular belt; this step is used to implement the function of the multi-sensor array module in the first embodiment, and will not be repeated here;

[0080] S200, extracting the geometric contour of the coal flow cross section in real time by using a coal flow edge detection algorithm that integrates RGB images and depth data; this step is used to implement the function of the edge computing unit in Example 1, and will not be repeated here;

[0081] S300, based on the geometric profile of the coal flow cross section and the conveyor belt speed sensor data, the instantaneous coal quantity is calculated by the integral method, and the material stacking density coefficient is dynamically corrected, and the instantaneous coal quantity is corrected according to the material stacking density coefficient; this step is used to realize the function of the coal quantity dynamic calculation module in the first embodiment, and will not be repeated here;

[0082] S400, set multi-level coal quantity thresholds, obtain the sampling period coal quantity according to the corrected instantaneous coal quantity, trigger the sound and light alarm, frequency conversion speed regulation signal or emergency stop command according to the comparison value of the sampling period coal quantity and the multi-level coal quantity threshold, and interlock control the conveyor drive device; this step is used to realize the function of the graded warning module in Example 1, and will not be repeated here.

[0083] As a preferred implementation of this embodiment, the 3D vision sensor adopts a binocular stereo camera array or a ToF laser scanner, which is installed above the opening side of the tubular belt to cover the entire cross-section of the conveyor belt at an inclination angle of 45-60°.

[0084] As a preferred implementation of this embodiment, adaptive coal flow region segmentation based on HSV color space;

[0085] Edge refinement extraction combining Canny operator and morphological processing;

[0086] The inner wall baseline of the tubular belt is fitted by the RANSAC algorithm, and the actual filling rate of the coal flow section is calculated to extract the geometric contour of the coal flow section.

[0087] As a preferred implementation mode of this embodiment, in the step of correcting the instantaneous coal quantity, the tension sensor data is also integrated, a coal quantity-belt tension coupling model is established, multi-source data fusion is performed through Kalman filtering, and the instantaneous coal quantity calculation error is corrected based on the multi-source fusion data.

[0088] As a preferred implementation of this embodiment, the triggering condition of the emergency stop command is that the coal volume exceeds the threshold value for three consecutive sampling cycles and the gradient change rate is greater than 5% / s, and the upstream feeding equipment is simultaneously linked to an emergency stop.

[0089] Embodiment five:

[0090] This embodiment proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the vision-based tubular belt conveyor coal flow monitoring and protection method as described in any embodiment of the present invention is implemented.

[0091] Embodiment six:

[0092] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the program is executed by a processor, the vision-based tubular belt conveyor coal flow monitoring and protection method as described in any embodiment of the present invention is implemented.

[0093] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent the existence of A alone, the existence of A and B at the same time, and the existence of B alone. Among them, A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b and c can be represented by: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, c can be single or multiple.

[0094] Those of ordinary skill in the art will appreciate that the various units and algorithm steps described in the embodiments disclosed herein can be implemented in a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0095] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0096] In several embodiments provided in the present application, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory; hereinafter referred to as: ROM), random access memory (Random Access Memory; hereinafter referred to as: RAM), disk or optical disk and other media that can store program codes.

[0097] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A vision-based tubular belt conveyor coal flow monitoring and protection system, characterized in that: include: Multi-sensor array module: comprising at least one set of 3D vision sensors and industrial cameras arranged longitudinally along the conveyor belt, for synchronously collecting stereoscopic cross-sectional images of the coal flow in the tubular belt; Edge computing unit: Built-in coal flow edge detection algorithm, which extracts the geometric contour of the coal flow section in real time by fusing RGB images and depth data; Coal quantity dynamic calculation module: Based on the geometric profile of the coal flow section and the conveyor belt speed sensor data, the instantaneous coal quantity is calculated using the integral method, and the material stacking density coefficient is dynamically corrected. The instantaneous coal quantity is corrected according to the material stacking density coefficient; Gradual warning module: set multi-level coal quantity thresholds, obtain the sampling period coal quantity according to the corrected instantaneous coal quantity, trigger the sound and light alarm, frequency conversion speed regulation signal or emergency stop command according to the comparison value between the sampling period coal quantity and the multi-level coal quantity thresholds, and interlock control the conveyor drive device.

2. The vision-based tubular belt conveyor coal flow monitoring and protection system according to claim 1 is characterized in that: The 3D vision sensor adopts a binocular stereo camera array or a ToF laser scanner, which is installed above the opening side of the tubular belt to cover the entire cross-section of the conveyor belt at an inclination angle of 45-60°.

3. The vision-based tubular belt conveyor coal flow monitoring and protection system according to claim 1, characterized in that: The coal flow edge detection algorithm includes: Adaptive coal flow region segmentation based on HSV color space; Edge refinement extraction combining Canny operator and morphological processing; The inner wall baseline of the tubular belt is fitted by the RANSAC algorithm, and the actual filling rate of the coal flow section is calculated to extract the geometric contour of the coal flow section.

4. The vision-based tubular belt conveyor coal flow monitoring and protection system according to claim 1, characterized in that: The coal quantity dynamic calculation module integrates the tension sensor data, establishes a coal quantity-belt tension coupling model, performs multi-source data fusion through Kalman filtering, and corrects the instantaneous coal quantity calculation error according to the multi-source fusion data.

5. The vision-based tubular belt conveyor coal flow monitoring and protection system according to claim 1, characterized in that: The triggering condition of the emergency stop command is that the coal volume exceeds the threshold value for three consecutive sampling cycles and the gradient change rate is greater than 5% / s, and the upstream feeding equipment is simultaneously linked to an emergency stop.

6. A vision-based method for monitoring and protecting coal flow in a tubular belt conveyor, characterized in that: The following steps are involved: A multi-sensor array module including at least one set of 3D vision sensors and industrial cameras arranged longitudinally along the conveyor belt is provided to synchronously collect stereoscopic cross-sectional images of the coal flow in the tubular belt; Through the coal flow edge detection algorithm that integrates RGB images and depth data, the geometric contour of the coal flow cross section is extracted in real time; Based on the geometric profile of the coal flow section and the data of the conveyor belt speed sensor, the instantaneous coal quantity is calculated by the integral method, and the material stacking density coefficient is dynamically corrected. The instantaneous coal quantity is corrected according to the material stacking density coefficient; Set multi-level coal quantity thresholds, obtain the sampling period coal quantity based on the corrected instantaneous coal quantity, trigger the sound and light alarm, frequency conversion speed regulation signal or emergency stop command based on the comparison value between the sampling period coal quantity and the multi-level coal quantity thresholds, and interlock control the conveyor drive device.

7. The visual-based tubular belt conveyor coal flow monitoring and protection method according to claim 6 is characterized in that: The 3D vision sensor adopts a binocular stereo camera array or a ToF laser scanner, which is installed above the opening side of the tubular belt to cover the entire cross-section of the conveyor belt at an inclination angle of 45-60°.

8. The visual-based tubular belt conveyor coal flow monitoring and protection method according to claim 6, characterized in that: The coal flow edge detection algorithm includes: Adaptive coal flow region segmentation based on HSV color space; Edge refinement extraction combining Canny operator and morphological processing; The inner wall baseline of the tubular belt is fitted by the RANSAC algorithm, and the actual filling rate of the coal flow section is calculated to extract the geometric contour of the coal flow section.

9. The visual-based tubular belt conveyor coal flow monitoring and protection method according to claim 6, characterized in that: In the step of correcting the instantaneous coal quantity, the tension sensor data is also integrated to establish a coal quantity-belt tension coupling model, and multi-source data fusion is performed through Kalman filtering. The instantaneous coal quantity calculation error is corrected based on the multi-source fusion data.

10. The visual-based tubular belt conveyor coal flow monitoring and protection method according to claim 6, characterized in that: The triggering condition of the emergency stop command is that the coal volume exceeds the threshold value for three consecutive sampling cycles and the gradient change rate is greater than 5% / s, and the upstream feeding equipment is simultaneously linked to an emergency stop.

Images