Belt conveyor material flow real-time analysis system based on active 4D stereoscopic vision

Abstract

The invention relates to a belt conveyor material flow real-time analysis system based on active 4D stereoscopic vision. The system comprises a 3D laser vision module, wherein the 3D laser vision module comprises three groups of 3D high-frame-rate industrial cameras and corresponding laser emitters; the 3D high-frame-rate industrial cameras are arranged at the head part, the middle part and the tail part of a belt conveyor and are synchronously triggered in time; the motion compensation modules are arranged at the head and the tail of the belt conveyor; the data processing terminal is respectively connected with the 3D laser vision module and the motion compensation module through special cables; and the data display and control terminal is in communication connection with the data processing terminal and is used for receiving the processing result and displaying the 4D dynamic model, the flow data and the alarm information in real time. By means of the software and hardware collaborative design and algorithm innovation, high-precision, real-time and non-contact three-dimensional visual monitoring and intelligent analysis of the material flow of the belt conveyor are achieved.

Description

Technical Field

[0001] This invention relates to the field of industrial inspection technology, and in particular to a real-time material flow analysis system for belt conveyors based on active 4D stereo vision. Background Technology

[0002] In existing belt conveyor systems, material flow detection mainly employs contact-based or non-visual methods such as belt scales and flow switches. Traditional weighing methods (such as belt scales) are easily affected by factors such as belt tension and vibration, resulting in low accuracy and high maintenance requirements. Flow switches and similar devices only provide switching signals and cannot achieve continuous and accurate flow measurement.

[0003] Existing visual inspection solutions are mostly passive vision technologies, relying on the natural texture of the material surface. Their effectiveness is poor or even fails on low-texture materials such as coal. Some laser scanning solutions are mostly single-line scans, which are prone to motion blur under high-speed conveyor belt operation, leading to large measurement errors (typically greater than 10mm). Furthermore, some solutions using TOF (Time-of-Flight) cameras are significantly affected by ambient light, especially under sunlight conditions, where the error can increase sharply to over 5%, making it difficult to meet the high-precision measurement requirements of industrial sites.

[0004] Therefore, there is an urgent need for a real-time material flow analysis system for belt conveyors that can overcome the above-mentioned shortcomings and achieve high precision, non-contact operation, and strong anti-interference capabilities. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing technologies by providing a real-time material flow analysis system for belt conveyors based on active 4D stereo vision. This system combines multiple synchronously triggered 3D laser vision modules with motion compensation modules to achieve precise 4D (three-dimensional space + time) dynamic modeling of the material profile, thereby enabling high-precision, non-contact real-time analysis and alarm of material flow.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A real-time material flow analysis system for belt conveyors based on active 4D stereo vision includes:

[0008] 3D laser vision module: includes three sets of time-synchronized 3D high frame rate industrial cameras and corresponding laser emitters deployed at the head, middle and tail of the belt conveyor, used to project structured lasers onto the belt cross-section in a time sequence and acquire continuous laser stripe images.

[0009] Motion compensation module: Deployed at the head and tail of the belt conveyor, it is used to collect and correct the belt's running speed in real time, providing speed data in the time dimension;

[0010] Data processing terminal: Connected to the 3D laser vision module and motion compensation module via dedicated cables, it receives continuous laser stripe images and velocity data. By fusing the fixed spatial positional relationship of the three vision modules with the time difference of material movement between adjacent modules, it generates and spatiotemporally stitches three-dimensional point clouds, realizes 4D dynamic modeling of material outline, and calculates real-time volumetric flow rate.

[0011] Data display and control terminal: Communicates with the data processing terminal to receive processing results and display 4D dynamic models, traffic data, and alarm information in real time.

[0012] In particular, the 3D high frame rate industrial camera adopts a global exposure mode with an exposure time of less than 1ms, and the laser emitter adopts a dual-line laser scanning method.

[0013] Specifically, the conveyor belt frame is equipped with brackets at the head, middle and tail sections, and 3D laser vision modules are mounted on the brackets. The field of view of the three sets of 3D laser vision modules covers the head, middle and tail of the conveyor belt, and the detection blind spots are eliminated by the fusion of multi-view point clouds in the spatiotemporal dimension.

[0014] In particular, the motion compensation module detects changes in belt tension using non-contact sensors and dynamically corrects the speed data.

[0015] Specifically, the data processing terminal executes the following processing flow:

[0016] S1. Based on the laser stripe images synchronously acquired by three sets of 3D laser vision modules, and combined with their spatial position and time difference information, a three-dimensional point cloud is generated through an improved stereo matching algorithm.

[0017] S2. Using the belt speed data provided by the motion compensation module, the point cloud is spatiotemporally stitched together using the optimized ICP algorithm to form a continuous 4D model.

[0018] S3. Calculate the cross-sectional area of ​​the material in the current belt section based on the spliced ​​4D model, and accumulate the volumetric flow rate through integral calculation.

[0019] In particular, the improved stereo matching algorithm is the SGBM algorithm that incorporates laser stripe gradient constraints.

[0020] In particular, the data display and control terminal is integrated with a big data analysis unit, which can dynamically set traffic alarm thresholds based on historical data.

[0021] In particular, the data processing terminal has a built-in self-calibration algorithm that can dynamically adjust the laser power and camera exposure parameters according to the ambient light intensity.

[0022] Specifically, when the data processing terminal performs point cloud processing and stitching in S1 and S2, it further adopts the following processing architecture:

[0023] Data preprocessing layer: performs time synchronization and coordinate unification on the collected data;

[0024] Core stitching layer: Based on motion constraint compensation and combined with laser feature weighting, estimation is performed through rigid body transformation. Point cloud registration is performed, where T is the rigid body transformation matrix; p i Let q be the i-th 3D point in the target point cloud; i Let i be the i-th 3D point in the source point cloud;

[0025] Post-processing layer: Performs statistical outlier filtering and overlapping region fusion on the registered point cloud to output a global point cloud model.

[0026] Specifically, the alarm control process is as follows:

[0027] Before image acquisition, determine whether the synchronization trigger signal is valid;

[0028] If the above is ineffective, the system returns to a waiting state;

[0029] If effective, process steps S1 to S3 are executed, and data anomalies during the process are detected and alarmed in real time.

[0030] The beneficial effects of this invention are:

[0031] High precision and non-contact measurement: It adopts an active structured laser light source to overcome the detection challenge of low-texture materials (such as coal), and combined with a high frame rate industrial camera and motion compensation, it achieves sub-pixel level point cloud accuracy with a system error of less than 1%.

[0032] 4D dynamic modeling eliminates blind spots: By synchronizing and fusing data from three sets of vision modules (head, middle, and tail), a 4D dynamic model of the material is constructed, effectively eliminating blind spots in single-point measurement and improving the completeness and accuracy of the measurement.

[0033] Strong anti-interference capability: The built-in self-calibration algorithm can dynamically adjust system parameters according to the ambient light intensity, and adopts non-contact speed correction, which effectively suppresses the system error introduced by belt tension changes and ambient light changes (controlled within ±2%), and has good stability.

[0034] High level of intelligence: The data processing terminal integrates improved stereo matching (SGBM) and point cloud stitching (ICP) algorithms. The data display and control terminal has big data analysis capabilities based on historical data, and can realize dynamic setting of traffic thresholds and intelligent alarms, thereby improving the automation level of the system.

[0035] Easy to maintain and long service life: The entire system adopts non-contact detection, avoiding direct contact with materials and belts, reducing equipment wear, and lowering maintenance costs and downtime. Attached Figure Description

[0036] Figure 1 This is a diagram of the 3D laser vision module and a schematic diagram of the deployment location of the motion compensation module of the present invention;

[0037] Figure 2 This is a system module diagram of the present invention;

[0038] Figure 3 This is a flowchart of the data processing terminal of the present invention;

[0039] Figure 4 This is the alarm flowchart of the present invention;

[0040] Figure 5 This is a diagram of the processing architecture for point cloud processing and stitching in this invention.

[0041] In the diagram: 1-3D laser vision module; 11-3D high frame rate industrial camera; 12-laser emitter; 13-stand; 2-motion compensation module; 3-data processing terminal; 4-data display and control terminal; 5-belt conveyor;

[0042] The following will describe in detail, with reference to the accompanying drawings, embodiments of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to embodiments:

[0044] like Figures 1-5 As shown, the real-time material flow analysis system for belt conveyors based on active 4D stereo vision includes a 3D laser vision module 1, a motion compensation module 2, a data processing terminal 3, and a data display and control terminal 4.

[0045] The 3D laser vision module 1 includes three sets of time-synchronized 3D high-frame-rate industrial cameras 11 and corresponding laser emitters 12 deployed at the head, middle, and tail of the conveyor belt 5. These cameras project structured laser light onto the conveyor belt cross-section in a time sequence and acquire continuous laser stripe images. The 3D high-frame-rate industrial cameras 11 employ active laser emission technology and dual-line laser scanning to acquire dual-line laser stripe patterns. Specifically, the laser emitters 12 project structured laser light onto the conveyor belt cross-section in a time sequence, while the 3D high-frame-rate industrial cameras 11 acquire continuous laser stripe images. The 3D high-frame-rate industrial cameras 11 use a global exposure mode with an exposure time of less than 1ms to effectively suppress image blurring caused by high-speed conveyor belt movement. The laser emitters 12 employ a dual-line laser scanning method to form richer feature information on low-texture material surfaces. The head, middle and tail of the belt conveyor 5 are respectively equipped with brackets 13. The 3D laser vision module 1 is installed on the brackets 13, and the field of view of the three sets of 3D laser vision modules 1 covers the head, middle and tail of the belt. The detection blind spot is eliminated by the fusion of multi-view point clouds in the spatiotemporal dimension.

[0046] By employing a spatial layout of the head, middle, and tail points and high-precision time synchronization, a 4D (three-dimensional space + time) measurement network is constructed, fundamentally eliminating blind spots caused by changes in material accumulation shape or belt vibrations in single-point measurements. Global exposure and microsecond-level exposure time ensure that clear, unblurred laser stripe images can be captured even under high-speed belt movement, laying the foundation for subsequent high-precision point cloud reconstruction. Dual-line laser scanning enhances the feature information on the surface of low-texture materials such as coal, improving the reliability and accuracy of image matching.

[0047] Motion compensation module 2 is deployed at the head and tail of belt conveyor 5 to collect and correct the belt's running speed in real time, providing speed data in the time dimension; motion compensation module 2 detects belt tension changes through non-contact sensors and dynamically corrects the speed data.

[0048] By employing non-contact sensing and dynamic correction technology, speed measurement errors (typically ±2% or higher) caused by belt slippage, elongation, or load variations can be effectively eliminated, maintaining extremely high speed measurement accuracy. This is a crucial prerequisite for achieving accurate volumetric flow rate accumulation, ensuring the reliability of flow calculation results.

[0049] The data processing terminal 3 is connected to the 3D laser vision module 1 and the motion compensation module 2 via dedicated cables. It is used to receive continuous laser stripe images and velocity data. By fusing the fixed spatial positional relationship of the three vision modules with the time difference of the material's movement between adjacent modules, it generates and spatiotemporally stitches a 3D point cloud, realizing 4D dynamic modeling of the material's outline and calculating real-time volumetric flow rate. The data processing terminal has a built-in self-calibration algorithm that can dynamically adjust the laser power and camera exposure parameters according to the ambient light intensity, ensuring that the point cloud accuracy error is less than 1% under sunlight conditions, in order to maintain the best acquisition state.

[0050] Data processing terminal 3 executes the following processing flow:

[0051] S1. Based on the laser stripe images synchronously acquired by three sets of 3D laser vision modules 1, and combined with their spatial position and time difference information, a three-dimensional point cloud is generated by an improved stereo matching algorithm; wherein, the improved stereo matching algorithm is the SGBM algorithm that introduces laser stripe gradient constraints, which significantly improves the matching accuracy in low-texture areas.

[0052] Specific methods for generating 3D point clouds include:

[0053] An improved SGBM stereo matching algorithm is executed on the laser stripe images synchronously acquired by three sets of 3D laser vision modules 1:

[0054] A laser stripe gradient constraint term is introduced into the traditional SGBM cost calculation to enhance the matching of low-texture regions.

[0055] By combining the fixed spatial positional relationship of the three modules and the time difference of material movement between adjacent modules, a spatiotemporal joint matching cost function is constructed to output three-dimensional point cloud coordinates with sub-pixel accuracy.

[0056] S2. Using the belt speed data provided by the motion compensation module 2, the point cloud is spatiotemporally stitched together using the optimized ICP algorithm to form a continuous 4D model.

[0057] S3. Calculate the current cross-sectional area of ​​the material on the belt according to the spliced ​​4D model, and accumulate the volumetric flow rate through integration calculation; where the current cross-sectional area S=∫h(x)dx (h is the material height distribution), and the accumulated volume V+=S×v×Δt (v is the belt speed).

[0058] This process forms a complete closed loop of 4D modeling and flow calculation. The improved SGBM algorithm in stage S1 specifically addresses the matching challenge for low-texture materials. Stage S2 combines high-precision velocity data for spatiotemporal stitching, synthesizing discrete 3D snapshots into a continuous 4D dynamic model, realistically reproducing the material's motion state. Finally, stage S3 calculates the volumetric flow rate through integration, achieving a level of accuracy far exceeding traditional methods.

[0059] When data processing terminal 3 performs point cloud processing and stitching in S1 and S2, it further adopts the following processing architecture:

[0060] Data preprocessing layer: performs time synchronization and coordinate unification on the collected data;

[0061] Core stitching layer: Based on motion constraint compensation and combined with laser feature weighting, estimation is performed through rigid body transformation. Point cloud registration is performed, where T is the rigid body transformation matrix; p i Let q be the i-th 3D point in the target point cloud; i Let i be the i-th 3D point in the source point cloud;

[0062] Post-processing layer: Performs statistical outlier filtering and overlapping region fusion on the registered point cloud to output a global point cloud model.

[0063] This architecture modularizes and refines the complex point cloud processing workflow. The preprocessing layer ensures data synchronization and coordinate consistency. The core stitching layer innovatively combines "motion constraint compensation" and "laser feature weighting" into the registration algorithm, significantly improving the accuracy and efficiency of stitching by utilizing prior system information. The post-processing layer optimizes point cloud quality through filtering and fusion, providing a reliable data foundation for accurate measurement.

[0064] The data display and control terminal 4 communicates with the data processing terminal 3 to receive processing results and display the 4D dynamic model, flow data, and alarm information in real time. The data display and control terminal 4 integrates a big data analysis unit, which can dynamically set flow alarm thresholds based on historical data. Big data analysis enables the alarm thresholds to adapt to changes in operating conditions, reducing false alarms and missed alarms.

[0065] The alarm control process is as follows:

[0066] Before image acquisition, determine whether the synchronization trigger signal is valid;

[0067] If the above is ineffective, the system returns to a waiting state;

[0068] If effective, process steps S1 to S3 are executed, and data anomalies during the process are detected and alarmed in real time.

[0069] The complete alarm control logic ensures that the system performs measurements only when synchronization is effective and data is reliable. Combined with the aforementioned self-calibration function, the system can actively adapt to drastic changes in ambient light, such as from night to day, maintaining a measurement accuracy of <1% under outdoor sunlight conditions, demonstrating extremely strong environmental adaptability and stability.

[0070] Through the above-mentioned collaborative design of software and hardware and algorithm innovation, this invention realizes high-precision, real-time, non-contact three-dimensional visualization monitoring and intelligent analysis of material flow of belt conveyors.

[0071] In the description of this invention, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," "counterclockwise," "axial," "radial," and "circumferential" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0072] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection, an electrical connection, or a connection that allows communication between them; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise explicitly limited. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0073] The present invention has been described above by way of example. Obviously, the specific implementation of the present invention is not limited to the above-described manner. Any improvements made by adopting the inventive concept and technical solution of the present invention, or direct application to other occasions without modification, are all within the protection scope of the present invention.

Claims

1. A real-time material flow analysis system for belt conveyors based on active 4D stereo vision, characterized in that, include: 3D laser vision module (1): includes three sets of 3D high frame rate industrial cameras (11) and corresponding laser emitters (12) deployed at the head, middle and tail of the belt conveyor (5) to project structured laser onto the belt cross section in a time sequence and acquire continuous laser stripe images. Motion compensation module (2): Deployed at the head and tail of the belt conveyor (5) to collect and correct the running speed of the belt in real time and provide speed data in the time dimension; Data processing terminal (3): It is connected to the 3D laser vision module (1) and motion compensation module (2) respectively via dedicated cables. It is used to receive continuous laser stripe images and speed data. By fusing the fixed positional relationship of the three vision modules in space with the running time difference of the material between adjacent modules, it generates and spatiotemporally splices three-dimensional point clouds, realizes 4D dynamic modeling of material outline, and calculates real-time volume flow rate. Data display and control terminal (4): It is connected to the data processing terminal (3) to receive processing results and display 4D dynamic model, traffic data and alarm information in real time.

2. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 1, characterized in that, The 3D high frame rate industrial camera (11) adopts a global exposure mode with an exposure time of less than 1ms, and the laser emitter (12) adopts a dual-line laser scanning method.

3. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 1 or 2, characterized in that, The head, middle and tail of the belt conveyor (5) are respectively equipped with brackets (13), and the 3D laser vision module (1) is installed on the bracket (13). The field of view of the three sets of 3D laser vision modules (1) covers the head, middle and tail of the belt. The detection blind spot is eliminated by the fusion of multi-view point clouds in the spatiotemporal dimension.

4. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 1, characterized in that, The motion compensation module (2) detects belt tension changes through non-contact sensors and dynamically corrects speed data.

5. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 1, characterized in that, The data processing terminal (3) executes the following processing flow: S1. Based on the laser stripe images synchronously acquired by three sets of 3D laser vision modules (1), combined with their spatial position and time difference information, a three-dimensional point cloud is generated by an improved stereo matching algorithm. S2. Using the belt speed data provided by the motion compensation module (2), the point cloud is spatiotemporally stitched together using the optimized ICP algorithm to form a continuous 4D model. S3. Calculate the cross-sectional area of ​​the material in the current belt section based on the spliced ​​4D model, and accumulate the volumetric flow rate through integral calculation.

6. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 5, characterized in that, The improved stereo matching algorithm is the SGBM algorithm with laser stripe gradient constraints.

7. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 1, characterized in that, The data display and control terminal (4) is integrated with a big data analysis unit, which can dynamically set the traffic alarm threshold based on historical data.

8. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 1, characterized in that, The data processing terminal has a built-in self-calibration algorithm that can dynamically adjust the laser power and camera exposure parameters according to the ambient light intensity.

9. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 5, characterized in that, When the data processing terminal (3) performs point cloud processing and stitching in S1 and S2, it further adopts the following processing architecture: Data preprocessing layer: performs time synchronization and coordinate unification on the collected data; Core stitching layer: Based on motion constraint compensation and combined with laser feature weighting, estimation is performed through rigid body transformation. Point cloud registration is performed, where T is the rigid body transformation matrix; p i Let q be the i-th 3D point in the target point cloud; i Let i be the i-th 3D point in the source point cloud; Post-processing layer: Performs statistical outlier filtering and overlapping region fusion on the registered point cloud to output a global point cloud model.

10. The real-time material flow analysis system for belt conveyors based on active 4D stereo vision according to claim 5, characterized in that, The alarm control process is as follows: Before image acquisition, determine whether the synchronization trigger signal is valid; If the above is ineffective, the system returns to a waiting state; If effective, process steps S1 to S3 are executed, and data anomalies during the process are detected and alarmed in real time.

Images