A deep learning-based coal quantity monitoring method and system for a scraper conveyor

Abstract

The application discloses a kind of based on deep learning's coal flow detection method and system of scraper conveyor, and the application obtains coal flow image sequence by multi-view image acquisition, establishes unified three-dimensional coordinate system by time synchronization, space calibration, constructs multi-angle projection data set;Again, three-dimensional voxel grid is formed by coal flow region voxel dispersion, combined with back projection constraint and depth feature guided iteration inversion obtains coal flow three-dimensional distribution probability field;Rely on the correction of conveying motion and mass continuity constraint to obtain dynamic three-dimensional density field, finally, the cumulative coal quantity is calculated by cross-sectional area division combined with conveying speed, coal density.This application solves the problem of low precision, easy to be shielded interference, big detection result fluctuation of traditional two-dimensional detection method, can accurately restore coal flow three-dimensional distribution, effectively suppresses detection error and data jump, adapts complex mine, and improves the stability and accuracy of coal quantity detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent detection of coal mine conveying equipment, and in particular to a method and system for monitoring the amount of coal conveyed by a scraper conveyor based on deep learning. Background Technology

[0002] As a key conveying equipment in underground coal mine working faces, transfer points, and various bulk material conveying scenarios, scraper conveyors play a crucial role in the continuous transportation of coal. Their operating status and the amount of coal conveyed directly affect the accuracy of production organization, energy consumption management, equipment scheduling, and safety monitoring. With the trend towards intelligent, less-manned, and unmanned coal mines, how to accurately and continuously monitor the amount of coal conveyed by scraper conveyors in real time has become one of the key technical issues in the automation control and intelligent sensing of coal mine equipment.

[0003] In existing technologies, methods for detecting coal quantity in conveying equipment mainly focus on weighing detection, volume estimation detection, and detection methods based on single image processing. For belt conveyors, electronic belt scales, weighing idlers, or flow meters are commonly used to measure the conveying volume. However, these technologies mostly rely on a relatively stable conveying structure and a relatively regular material distribution. The operating environment of scraper conveyors differs significantly from that of belt conveyors. The conveying trough space is highly enclosed, and the coal flow exhibits obvious accumulation, tumbling, dispersion, and irregular fluctuation characteristics. It is also greatly affected by factors such as scraper traction, trough vibration, coal particle size differences, underground dust, water mist, and changes in lighting. Therefore, traditional coal quantity detection methods used in belt conveyor scenarios are difficult to directly apply to scraper conveyor coal conveying scenarios. Summary of the Invention

[0004] The present invention aims to at least partially solve one of the technical problems in the related art.

[0005] The main objective of this invention is to provide a deep learning-based method for detecting the amount of coal conveyed by a scraper conveyor.

[0006] Another objective of this invention is to propose a deep learning-based coal conveying quantity detection system for scraper conveyors.

[0007] To achieve the above objectives, a first aspect of the present invention proposes a deep learning-based method for detecting coal quantity in a scraper conveyor, comprising:

[0008] Image acquisition units with different perspectives are arranged circumferentially in the scraper conveyor channel to acquire continuous image sequences of coal flow from different perspectives. Time synchronization and spatial calibration processing are performed on continuous image sequences from different perspectives to establish a unified three-dimensional coordinate system and preprocess to obtain standardized projected image data; Based on standardized projection image data, we extract the projection contour information and grayscale distribution information of coal flow from various perspectives to construct a multi-angle projection dataset. The multi-angle projection dataset is mapped to a unified three-dimensional coordinate system, and the coal flow area in the scraper conveyor channel is spatially divided by voxel discretization to form a three-dimensional discrete voxel mesh. Based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel grid, the occupancy probability of each voxel is iteratively inverted using the back projection constraint and depth feature guidance method to obtain the three-dimensional distribution probability field of coal flow. Introducing motion constraints and mass continuity constraints in the scraper conveying direction into the three-dimensional probability field of coal flow, and performing spatiotemporal consistency correction on the voxel distribution at adjacent moments, a dynamic three-dimensional density field of coal flow is obtained. The cross-sectional area of ​​the three-dimensional density field of the dynamic coal flow is integrated along the conveying direction of the scraper conveyor, and the volumetric flow rate of the coal flow per unit time is calculated in combination with the chain running speed. Based on the volumetric flow rate of the coal flow and the preset coal density parameters, the mass flow rate of the coal flow is calculated, and the mass flow rate over a continuous time period is accumulated to obtain the cumulative coal quantity conveyed by the scraper conveyor and output it.

[0009] In one embodiment of the present invention, the process of obtaining a continuous image sequence includes: Based on the structural dimensions of the scraper conveyor's central trough, the coal flow trajectory, and the effective observation area, image acquisition units are installed at two locations above, to the side, and to the side of the conveying channel, forming a multi-view observation layout that has an overlapping relationship with the surface morphology and boundary contour of the coal flow. Each image acquisition unit is rigidly connected by an independent mounting frame and maintains a constant relative installation position. Based on the continuous acquisition of coal flow images by each image acquisition unit at a preset sampling frequency during the operation of the scraper conveyor, a continuous image sequence reflecting the spatial distribution of coal flow from different perspectives is obtained.

[0010] In one embodiment of the present invention, the process of obtaining the standardized projection image data includes: Based on the acquisition time of continuous image sequences from different perspectives, image frames corresponding to the same transmission segment from different perspectives are time-aligned to obtain a time-synchronized multi-view image sequence. Spatial calibration is performed on each image acquisition unit to obtain the intrinsic and extrinsic parameters of each image acquisition unit; A unified three-dimensional coordinate system is established based on the external parameters of each image acquisition unit; Noise suppression processing is performed on the time-synchronized multi-view image sequence to remove image noise introduced by dust disturbance, light fluctuation and equipment vibration, resulting in a denoised image sequence; The denoised image sequence is subjected to grayscale normalization to obtain a normalized image sequence; The normalized image sequence is subjected to geometric correction processing. The distortion in the images from each viewpoint is corrected by using the intrinsic parameters of each image acquisition unit to obtain the geometrically corrected image sequence. The geometrically corrected image sequence is mapped to a unified three-dimensional coordinate system to obtain standardized projected image data.

[0011] In one embodiment of the present invention, the process of constructing the multi-angle projection dataset includes: Standardized projection image data are organized according to viewpoint category and acquisition time to form a multi-view time-series image group. The coal flow region is segmented for each viewpoint image in the multi-view time-series image group to obtain a binary mask of the coal flow projection region corresponding to each viewpoint image. The outer boundary of the coal flow projection region under each viewpoint is extracted based on the binary mask of the coal flow projection region to obtain a set of boundary pixels that characterize the coal flow projection contour information. Based on the binary mask of the coal flow projection area corresponding to each viewpoint image, the pixel gray values ​​inside the coal flow projection area are read and collected to obtain the gray distribution set representing the gray distribution information of the coal flow under each viewpoint. The contour feature parameters of the coal flow projection area under each viewpoint are calculated based on the boundary pixel set, and the gray-level feature parameters of the coal flow projection area under each viewpoint are calculated based on the gray-level distribution set. By combining the set of boundary pixels, grayscale distribution, contour feature parameters, and grayscale feature parameters corresponding to each viewpoint at the same acquisition time in a unified data format, a multi-angle projection dataset corresponding to the acquisition time is constructed.

[0012] In one embodiment of the present invention, the formation process of the three-dimensional discrete voxel mesh includes: Using a unified three-dimensional coordinate system as a reference, the length, width, and height ranges of the scraper conveyor channel are obtained. Based on the preset spatial resolution in the length direction, width direction, and height direction, the ranges in the length direction, width direction, and height direction are discretized to obtain the number of voxels in the length direction, width direction, and height direction of the three-dimensional discrete voxel mesh. Based on the number of voxels in the length direction, width direction, and height direction, the coal flow area in the scraper conveyor channel is spatially divided to generate voxel units. The projection contour information and grayscale distribution information of each viewpoint in the multi-angle projection dataset are mapped to a unified three-dimensional coordinate system, and the projection correspondence between image pixels and voxel units under each viewpoint is established based on the spatial position parameters and imaging direction parameters of the image acquisition units of each viewpoint. For each viewpoint, a corresponding view cone constraint region is formed on the projected contour boundary. The view cone constraint regions under each viewpoint are spatially intersected in a unified three-dimensional coordinate system to obtain the initial coal flow candidate region corresponding to the multi-angle projection dataset. Voxel elements located within the initial coal flow candidate region are assigned candidate occupancy identifiers to form a three-dimensional discrete voxel mesh corresponding to the coal flow region within the scraper conveyor channel.

[0013] In one embodiment of the present invention, the process of obtaining the three-dimensional distribution probability field of the coal flow includes: Based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel mesh, the correspondence between the voxel unit and the pixel position of the projection plane under each viewpoint is established, and the occupancy probability of each voxel unit is calculated by forward projection according to the correspondence to obtain the estimated projection value. The estimated projection values ​​obtained from the forward projection calculation are compared with the actual projection values ​​in the multi-angle projection dataset to calculate the projection error at each viewpoint. The depth feature information corresponding to each viewpoint is extracted from the multi-angle projection dataset, and the depth feature information is mapped to the unified three-dimensional coordinate system to obtain the voxel feature vector corresponding to each voxel unit. Generate feature guidance terms for each voxel unit based on voxel feature vectors; The projection error is used as the back-projection constraint term, and the deviation between the occupancy probability of each voxel and the feature guidance term is used as the feature guidance constraint term. The back-projection constraint term and the feature guidance constraint term are combined to form a joint objective function. The voxel occupancy probability field in the next iteration is obtained by iteratively updating the occupancy probability of each voxel unit based on the joint objective function. When the change in the joint objective function is less than or equal to the preset convergence threshold between two adjacent iterations, the iterative update of the occupancy probability of each voxel unit is stopped, and the converged voxel occupancy probability field is determined as the three-dimensional distribution probability field of the coal flow.

[0014] In one embodiment of the present invention, the process of obtaining the dynamic three-dimensional density field of coal flow includes: The three-dimensional distribution probability field of coal flow is defined as a probability distribution composed of the occupancy probability of each voxel unit, and the running speed of the scraper conveyor chain at the corresponding time and the time interval between adjacent time moments are obtained. Based on the chain running speed and time interval, calculate the theoretical displacement distance of the coal flow along the scraper conveying direction between adjacent moments; Based on the theoretical displacement distance and the voxel size of the three-dimensional discrete voxel mesh in the scraper conveying direction, the amount of voxel migration of the coal flow along the scraper conveying direction between adjacent time points is determined. Based on the voxel migration amount, the occupancy probability of each voxel unit at the current time is displaced and propagated along the scraper conveying direction, and the occupancy probability of the voxel unit at the upstream position is transferred to the voxel unit at the corresponding position at the next time, so as to obtain the predicted occupancy probability field at the next time. By using the continuity constraint of quality, the predicted occupancy probability field is compared with the probability field of the three-dimensional distribution of coal flow at the next moment on a volume-by-volume basis, and the continuity constraint error term is calculated. Based on the continuity constraint error term, the spatiotemporal consistency correction of the occupancy probability of each voxel unit in the next moment is performed to obtain the corrected voxel occupancy probability field, and the dynamic coal flow three-dimensional density field is calculated based on the corrected voxel occupancy probability field.

[0015] In one embodiment of the present invention, the calculation process of the coal flow volumetric flow rate includes: In a unified three-dimensional coordinate system, the conveying direction of the scraper conveyor is taken as the flow calculation direction. A calculation section perpendicular to the conveying direction of the scraper conveyor is selected, and the three-dimensional density field of dynamic coal flow is discretized into cross-sectional voxel elements on the calculation section to form a discrete cross-sectional region. The density values ​​of each cross-sectional voxel element within the discrete cross-sectional region are calculated by cross-sectional integration to obtain the area-weighted density value of the coal flow at the corresponding time point. Obtain the scraper conveyor chain running speed that matches the calculation section and the corresponding time, and multiply the coal flow area weighted density value by the scraper conveyor chain running speed to obtain the coal flow volume flow rate per unit time through the calculation section at the corresponding time.

[0016] In one embodiment of the present invention, the process of obtaining and outputting the cumulative coal quantity conveyed by the scraper conveyor includes: The coal flow mass flow rate at the corresponding time is calculated based on the coal flow volume flow rate and the preset coal density parameters, so as to obtain the coal flow mass flow rate passing through the calculation section per unit time. The coal flow mass flow rate within a continuous time range is discretized by dividing the continuous time into discrete time periods and obtaining the coal flow mass flow rate data corresponding to each discrete time period. The cumulative mass flow rate of coal in each discrete time period is calculated to obtain the cumulative mass of coal in the continuous time range. The cumulative coal flow quality is correlated with the corresponding time information to generate and output the cumulative coal quantity data of the scraper conveyor.

[0017] To achieve the above objectives, a second aspect of the present invention provides a deep learning-based coal quantity detection system for scraper conveyors, which executes the deep learning-based coal quantity detection method for scraper conveyors described in any one of the first aspects of the invention, including: The image acquisition unit is used to acquire a continuous sequence of images of coal flow from different perspectives. The preprocessing unit is used to perform time synchronization, spatial calibration and preprocessing on continuous image sequences to obtain standardized projected image data and establish a unified three-dimensional coordinate system. The projection feature extraction unit is used to extract projection contour information and grayscale distribution information from standardized projection image data to construct a multi-angle projection dataset. The voxel reconstruction unit is used to map the multi-angle projection dataset to a unified three-dimensional coordinate system and to discretize the coal flow area in the scraper conveyor channel to form a three-dimensional discrete voxel mesh. The probability inversion unit is used to iteratively invert the occupancy probability of each voxel based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel grid, using inversion constraints and depth feature guidance to obtain the three-dimensional distribution probability field of coal flow. The spatiotemporal correction unit is used to introduce motion constraints and mass continuity constraints in the scraper conveying direction into the three-dimensional distribution probability field of coal flow, and to perform spatiotemporal consistency correction on the voxel distribution at adjacent moments to obtain the dynamic three-dimensional density field of coal flow. The flow calculation unit is used to perform cross-sectional integration of the three-dimensional density field of dynamic coal flow along the conveying direction of the scraper conveyor, and calculate the volumetric flow rate of coal flow per unit time in combination with the chain running speed. The coal quantity output unit is used to calculate the coal mass flow rate based on the coal volume flow rate and the preset coal density parameters, and to accumulate the mass flow rate over a continuous time period to obtain and output the cumulative coal quantity conveyed by the scraper conveyor.

[0018] The embodiments of the present invention have the following beneficial effects: 1. This invention uses a circumferential multi-view image acquisition unit for the scraper conveyor channel, combined with time synchronization, spatial calibration, and unified three-dimensional coordinate system mapping, to achieve multi-view collaborative perception of coal flow. By extracting coal flow contour and grayscale features through standardized multi-angle projection datasets, it can effectively eliminate detection deviations caused by single-view coal flow occlusion, boundary loss, and morphological distortion, and significantly improve the integrity and reliability of coal flow spatial information acquisition and detection.

[0019] 2. This invention constructs a three-dimensional voxel grid by voxel discretization, integrates multi-angle projection data, and uses back-projection constraints combined with depth feature iterative inversion to obtain the three-dimensional distribution probability field of coal flow. It breaks through the limitations of existing two-dimensional image estimation of coal quantity, can completely reconstruct the three-dimensional spatial occupancy state of coal flow, truly restore the three-dimensional distribution characteristics of coal flow, and significantly improve the three-dimensional characterization capability and measurement accuracy of coal quantity detection.

[0020] 3. This invention introduces motion constraints and mass continuity constraints of scraper conveying to perform spatiotemporal consistency correction on voxel distribution, generating a dynamic three-dimensional density field of coal flow. This conforms to the objective laws of scraper conveyor operating speed and coal flow migration, suppressing defects such as local jumps and random fluctuations in single-frame detection. This not only improves the stability of instantaneous coal quantity detection, but also effectively improves the accuracy of cumulative coal quantity calculation over a period of time.

[0021] 4. This invention is based on the dynamic three-dimensional density field section integration combined with the conveying speed to solve the volumetric flow rate, match the coal density to convert the mass flow rate and accumulate it over time to obtain the total coal conveying volume. The metering process relies on multi-dimensional real working condition parameters such as three-dimensional spatial distribution, dynamic density and conveying speed, avoiding the error accumulation problem caused by the rough empirical coefficients and the substitution of local features for global state in traditional methods. The calculation results of coal flow volumetric flow rate, mass flow rate and cumulative coal volume are more in line with the actual conveying conditions on site.

[0022] 5. This invention is a non-contact detection solution, which is suitable for complex underground working conditions such as high dust and strong vibration of scraper conveyors, and has excellent resistance to environmental interference. It can accurately restore the three-dimensional spatial distribution of coal flow, and the detection values ​​are dynamically stable with small long-term cumulative measurement errors. It can meet the needs of various engineering applications such as continuous online detection of coal quantity, production scheduling, equipment linkage control, and real-time monitoring of conveying conditions in intelligent coal mine conveying systems. Attached Figure Description

[0023] The above-described and additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which: Figure 1 A flowchart illustrating a deep learning-based method for monitoring coal conveying volume using a scraper conveyor, as provided in an embodiment of the present invention. Figure 2 A schematic flowchart illustrating the construction process of a three-dimensional discrete voxel mesh provided in an embodiment of the present invention; Figure 3 This is a schematic diagram illustrating the process of constructing a three-dimensional probability field for coal flow provided in an embodiment of the present invention. Detailed Implementation

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] In existing technologies, coal conveying quantity detection in scraper conveyors mainly employs contact weighing, single-view image estimation, or conversion methods based on empirical formulas. While contact weighing can directly acquire some load information, the sensors and installation structures are prone to wear, drift, and failure in the high-vibration, high-impact, high-dust, and high-wear working environment of scraper conveyors. This results in poor long-term operational stability and high installation and maintenance difficulty, making it difficult to meet the reliability requirements of continuous underground operation. Single-view image estimation methods typically approximate calculations based on the coal flow surface contour, projected area, or grayscale characteristics. While this can reduce mechanical wear issues associated with contact measurements to some extent, the coal flow in the scraper conveyor channel is subject to obstruction, uneven accumulation, blurred boundaries, and dynamic tumbling. A single viewpoint cannot fully reflect the true three-dimensional spatial distribution of the coal flow, making the detection results susceptible to limitations in perspective and environmental interference. Conversion methods based on empirical formulas are simple to implement, but they are less adaptable to changes in operating conditions. When coal particle size, accumulation morphology, conveying speed, and lighting and dust conditions change, calculation errors tend to increase, leading to insufficient detection accuracy and stability.

[0027] Most existing technologies remain at the level of two-dimensional feature extraction or single-moment estimation, lacking the technical means to unify multi-view observation information into a single spatial coordinate framework for fusion processing, and also lacking the ability to express the spatial occupancy state of coal flow within the scraper conveyor channel at the voxel level. In other words, existing solutions can usually only estimate the "visible surface" and are difficult to reconstruct the "actually occupied three-dimensional space." Therefore, in cases of severe local obstruction of the coal flow, uneven coal distribution, or significant dynamic changes, the coal quantity detection results are prone to large deviations. In addition, existing technologies do not make sufficient use of the time dimension, often ignoring the objective laws of the scraper conveyor chain running speed, the time interval between adjacent moments, and the continuous migration of coal flow along the conveying direction. They lack a spatiotemporal consistency correction mechanism under motion constraints and quality continuity constraints, which leads to fluctuations and jumps in the detection results between adjacent moments, and the cumulative coal quantity error gradually increases.

[0028] Based on the shortcomings of the existing technology, the technical problems to be solved by the present invention are: how to provide a non-contact coal quantity detection system and method that is adaptable to dust interference and dynamic conveying environment under the complex working conditions of scraper conveyors; how to overcome the limitations of single-view and two-dimensional feature estimation to achieve an effective expression of the true three-dimensional spatial distribution of coal flow in the conveying channel; how to use multi-view image information and a unified three-dimensional coordinate system to establish the correspondence between coal flow projection data and spatial voxel units, thereby realizing the three-dimensional discrete reconstruction of the coal flow area; how to combine back projection constraints and depth feature guidance to iteratively invert the occupancy probability of each voxel to improve the accuracy of coal flow spatial reconstruction; how to further introduce motion constraints and mass continuity constraints in the scraper conveying direction to perform spatiotemporal consistency correction on the coal flow distribution at adjacent moments to obtain a more stable and more consistent dynamic three-dimensional density field of coal flow; and how to accurately calculate the volumetric flow rate, mass flow rate and cumulative coal quantity of the coal flow based on the dynamic three-dimensional density field of coal flow, thereby improving the accuracy, continuity and engineering applicability of coal quantity detection of scraper conveyors.

[0029] The following describes a method and system for monitoring coal quantity of a scraper conveyor based on deep learning, according to an embodiment of the present invention, with reference to the accompanying drawings.

[0030] Example 1 This embodiment provides a deep learning-based method for detecting coal quantity in a scraper conveyor. For example... Figure 1 As shown, the method includes the following steps: S1. Image acquisition units with different perspectives are arranged around the scraper conveyor channel to obtain a continuous image sequence of coal flow from different perspectives.

[0031] In the actual operating environment of the scraper conveyor channel, in order to achieve multi-dimensional perception of the coal flow morphology and its spatial distribution, this application embodiment sets up multiple sets of image acquisition units in the circumferential space of the conveying system, thereby constructing a stable and reliable multi-view visual observation system.

[0032] Specifically, this embodiment comprehensively plans the installation position of the image acquisition unit based on the structural dimension parameters of the scraper conveyor's central trough, the movement trajectory characteristics of the coal flow during the conveying process, and the spatial coverage of the effective observation area. The unit is arranged in representative spatial positions such as above, above and to the side of the conveying channel. In this embodiment, at least two of these positions are selected for combined deployment, thereby forming a multi-view observation structure with cross-coverage relationship for the undulation morphology, boundary contour changes and accumulation state of the coal flow surface.

[0033] In this configuration, each image acquisition unit in this embodiment is fixedly installed using an independently configured rigid mounting frame. This ensures the stability of its relative spatial position during long-term operation of the scraper conveyor, preventing viewpoint shifts caused by vibration, impact, or structural deformation. This guarantees spatial and temporal consistency among multi-view image data. In this embodiment, the image acquisition units do not share moving parts or flexible connection structures to reduce system coupling errors and improve the stability and reliability of multi-source data acquisition.

[0034] During operation, each image acquisition unit in this embodiment works synchronously or quasi-synchronously according to a preset sampling frequency to continuously acquire images of the continuously moving coal flow within the conveying channel, thereby obtaining continuous image sequence data from different perspectives. This image sequence can reflect the dynamic distribution characteristics of the coal flow during the conveying process from different spatial directions, including changes in the surface height of the coal flow, the trend of boundary expansion and contraction, and the evolution of local accumulation or sparse areas. Through the cross-overlay relationship between multi-view images, this embodiment provides basic data support for subsequent three-dimensional morphological reconstruction of the coal flow, state recognition, and intelligent analysis of the conveying process, thereby improving the overall perception and analysis accuracy of the scraper conveyor's conveying status.

[0035] S2. Perform time synchronization and spatial calibration processing on continuous image sequences from different perspectives, establish a unified three-dimensional coordinate system, and preprocess to obtain standardized projected image data.

[0036] In the process of coal flow detection and three-dimensional reconstruction in the scraper conveyor channel, in order to ensure the consistency of multi-view image data in the spatiotemporal dimension, the embodiments of this application first perform unified time synchronization and spatial calibration processing on the continuous image sequences obtained from different perspectives, thereby constructing a stable and reliable foundation for multi-source visual fusion.

[0037] Specifically, this application embodiment performs time alignment processing on image frames corresponding to the same transmission segment from different image acquisition units based on image acquisition time information from different image acquisition units, thereby eliminating time offset errors caused by acquisition delay or sampling frequency differences, and obtaining a multi-view image sequence after time synchronization, so that data from different viewpoints have comparability and consistency under the same time reference.

[0038] After time synchronization is completed, this embodiment performs spatial calibration processing on each image acquisition unit to obtain the intrinsic and extrinsic parameters of each image acquisition unit. The intrinsic parameters are used to describe the focal length, principal point position, and distortion characteristics of the imaging system, while the extrinsic parameters are used to describe the position and orientation of each image acquisition unit in the spatial coordinate system. Furthermore, this embodiment establishes a unified three-dimensional spatial coordinate system based on the extrinsic parameters of each image acquisition unit, thereby realizing the expression and fusion of multi-view observation data under a unified spatial reference system.

[0039] After obtaining the time synchronization and spatial calibration results, this embodiment of the application preprocesses the multi-view image sequence to improve data quality and the stability of subsequent analysis. First, noise suppression processing is performed on the image sequence to remove random noise interference introduced by factors such as dust disturbance, lighting fluctuations, and equipment vibration, thereby obtaining a denoised image sequence. Subsequently, grayscale normalization processing is performed on the denoised image sequence to eliminate brightness differences under different viewing angles and lighting conditions, ensuring consistency in the grayscale distribution of the image data.

[0040] Building upon this, the embodiments of this application further perform geometric correction processing on the normalized image sequence. The lens distortion in the image is corrected using the intrinsic parameters corresponding to each image acquisition unit, thereby eliminating geometric distortion caused by the imaging system and obtaining a geometrically corrected standardized image sequence. Finally, this embodiment maps the geometrically corrected multi-view image sequence to a unified three-dimensional spatial coordinate system, realizing the conversion of image data from two-dimensional pixel space to three-dimensional physical space, and obtaining standardized projected image data.

[0041] Through the aforementioned time synchronization, spatial calibration, and multi-level preprocessing procedures, the embodiments of this application effectively improve the spatiotemporal consistency and geometric consistency among multi-view image data, providing high-precision and high-stability basic data support for subsequent three-dimensional reconstruction of coal flow, state recognition, and intelligent analysis of the transportation process.

[0042] S3. Based on standardized projection image data, extract the projection contour information and grayscale distribution information of coal flow from various perspectives to construct a multi-angle projection dataset.

[0043] After obtaining standardized projection image data, this application further extracts the projection shape and image grayscale features of coal flow from different perspectives to construct a multi-angle projection dataset that can characterize the spatial distribution of coal flow.

[0044] Specifically, this embodiment first organizes the standardized projection image data according to the viewpoint category of the image acquisition unit and the corresponding acquisition time of the image. This ensures that projection images acquired by different image acquisition units at the same acquisition time belong to the same data group, forming a multi-view time-series image group. This processing method guarantees that the images from each viewpoint have a clear correspondence in both the time and spatial dimensions during the subsequent feature extraction process.

[0045] Subsequently, this embodiment performs coal flow region segmentation processing on each viewpoint image in the multi-view time-series image group, distinguishing the coal flow region in the image from the scraper conveyor trough, background structure, and other non-coal flow regions, obtaining a binary mask of the coal flow projection region corresponding to each viewpoint image. Based on this binary mask of the coal flow projection region, this embodiment further extracts the outer boundary of the coal flow projection region under each viewpoint, obtaining a set of boundary pixels used to characterize the coal flow projection contour information. This set of boundary pixels can reflect the external contour, accumulation edge, and cross-sectional change trend of the coal flow under the corresponding viewpoint.

[0046] Based on the extracted projection contour information, this embodiment of the application further reads, filters, and aggregates the pixel grayscale values ​​within the coal flow projection region using a binary mask corresponding to the coal flow projection region of each viewpoint image, obtaining a grayscale distribution set used to characterize the grayscale distribution information of the coal flow under each viewpoint. This grayscale distribution set can reflect the surface texture, brightness variations, particle accumulation differences, and local occlusion of the coal flow, providing auxiliary features for subsequent coal flow state judgment and three-dimensional morphological analysis.

[0047] Furthermore, this embodiment calculates the contour feature parameters of the coal flow projection area from each viewpoint based on the boundary pixel set. These contour feature parameters may include parameters such as projection area, circumscribed contour size, boundary length, contour centroid position, contour width variation, and boundary undulation degree. Simultaneously, this embodiment calculates the grayscale feature parameters of the coal flow projection area from each viewpoint based on the grayscale distribution set. These grayscale feature parameters may include parameters such as average grayscale value, grayscale variance, grayscale gradient, grayscale histogram distribution, and local grayscale variation characteristics.

[0048] Finally, this embodiment combines the boundary pixel set, grayscale distribution set, contour feature parameters, and grayscale feature parameters corresponding to each viewpoint at the same acquisition time according to a unified data format. This allows the projection information from different viewpoints to form a data structure that can be associated, called, and fused, thereby constructing a multi-angle projection dataset corresponding to the acquisition time. Through this multi-angle projection dataset, this embodiment can comprehensively characterize the projection contour, regional distribution, and grayscale variation characteristics of coal flow from multiple directions, providing a data foundation for subsequent three-dimensional volume estimation of coal flow, coal quantity identification, and transportation status analysis.

[0049] S4. Map the multi-angle projection dataset to a unified three-dimensional coordinate system, and spatially divide the coal flow area in the scraper conveyor channel by voxel discretization to form a three-dimensional discrete voxel mesh.

[0050] In this embodiment of the application, after obtaining the multi-angle projection dataset, in order to realize the three-dimensional expression of the coal flow spatial morphology, the multi-angle projection dataset is mapped to a unified three-dimensional coordinate system, and the coal flow area in the scraper conveyor channel is spatially divided by the voxel discretization method to establish a discrete spatial model suitable for three-dimensional reconstruction of coal flow, forming a three-dimensional discrete voxel mesh.

[0051] Specifically, refer to Figure 2 In this embodiment, the unified three-dimensional coordinate system established in step S2 is used as a spatial reference benchmark. Combined with the actual structural dimensions of the scraper conveyor channel, the spatial range of the conveyor channel along the length, width, and height directions is obtained. Among them, the length direction corresponds to the coal flow conveying direction, the width direction corresponds to the transverse dimension of the middle trough of the scraper conveyor, and the height direction corresponds to the coal flow accumulation height direction, thereby determining the three-dimensional spatial boundary of the area to be reconstructed.

[0052] After determining the spatial boundaries, this embodiment of the application uniformly discretizes the spatial range in the three directions according to the pre-set spatial resolutions in the length, width, and height directions, thereby calculating the number of voxels in the length, width, and height directions of the three-dimensional discrete voxel mesh. By adjusting the spatial resolution in each direction, the accuracy and computational efficiency of 3D reconstruction can be balanced according to different application scenarios to meet the needs of different coal flow detection tasks.

[0053] Subsequently, based on the number of voxels in each direction, this embodiment divides the coal flow area within the scraper conveyor channel into regular spatial divisions, dividing the entire three-dimensional space into multiple voxel units of uniform or preset size. Each voxel unit corresponds to a fixed spatial position in a unified three-dimensional coordinate system and has a unique spatial index used to describe the occupancy state of the coal flow at that spatial position, providing a unified data carrier for subsequent three-dimensional volume reconstruction.

[0054] After establishing the voxel mesh, this embodiment further maps the projection contour information and grayscale distribution information of each viewpoint in the multi-angle projection dataset to a unified three-dimensional coordinate system. Specifically, based on the spatial position parameters and imaging direction parameters corresponding to each image acquisition unit, and combined with the spatial calibration results obtained in step S2, a projection correspondence between image pixels of each viewpoint and three-dimensional voxel units is established, so that the projection information in the two-dimensional image can be accurately associated with the corresponding three-dimensional spatial position.

[0055] Furthermore, in this embodiment, the coal flow projection contour boundaries extracted from each viewpoint are used as constraints, and corresponding view cone constraint regions are constructed based on the imaging model of the image acquisition unit. The view cone constraint region represents the spatial constraint range extending from the optical center of the image acquisition unit through the coal flow projection contour boundaries into three-dimensional space, defining the possible spatial area where the coal flow may exist. Subsequently, the view cone constraint regions corresponding to multiple viewpoints are uniformly mapped to a three-dimensional coordinate system, and spatial intersection operations are performed on the view cone constraint regions formed by each viewpoint to obtain a common spatial region that simultaneously satisfies the projection constraints of multiple viewpoints, thereby obtaining the initial coal flow candidate region corresponding to the multi-angle projection dataset.

[0056] Finally, in this embodiment, each voxel element located within the initial coal flow candidate region is assigned a candidate occupancy identifier, marking it as a spatial voxel where coal flow may exist. Voxel elements located outside the candidate region remain in an unoccupied or background state. After the above processing, a three-dimensional discrete voxel grid corresponding to the spatial distribution of the coal flow at the current acquisition time is formed. This three-dimensional discrete voxel grid not only maintains the integrity of the coal flow spatial structure but also effectively integrates multi-view projection information, providing a unified spatial representation basis for subsequent voxel occupancy probability calculation, three-dimensional coal flow model reconstruction, and coal flow volume calculation.

[0057] S5. Based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel grid, the occupancy probability of each voxel is iteratively inverted using the back projection constraint and depth feature guidance method to obtain the three-dimensional distribution probability field of coal flow.

[0058] In this embodiment, after constructing a multi-angle projection dataset, in order to recover the three-dimensional distribution state of coal flow in the conveying space, based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel grid, a combination of back projection constraints and depth feature guidance is used to iteratively invert the occupancy probability of each voxel unit, and finally obtain a three-dimensional distribution probability field of coal flow that can characterize the spatial distribution state of coal flow.

[0059] Specifically, refer to Figure 3 In this embodiment, a three-dimensional discrete voxel grid covering the effective observation area of ​​coal flow is first established based on a unified three-dimensional coordinate system. The entire space to be reconstructed is divided into multiple voxel units of uniform or preset size. Each voxel unit corresponds to an occupancy probability to be solved, which represents the possibility that the spatial location is occupied by coal flow. In the initial state, the occupancy probability of each voxel unit can be assigned a uniform initial value, or it can be initialized based on historical coal flow distribution information or a priori models. This application does not limit this.

[0060] Subsequently, based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel mesh, this embodiment establishes a mapping relationship between voxel units and corresponding projection plane pixel positions at each viewpoint. Specifically, according to the extrinsic and intrinsic parameters of each image acquisition unit and a unified three-dimensional coordinate system, any voxel unit can be projected to a pixel position in the corresponding viewpoint image, thereby establishing a one-to-one or one-to-many correspondence between voxel units and projected pixels. Using the established projection relationship, the current occupancy probability of each voxel unit is forward-projected to obtain the estimated projection value corresponding to each viewpoint. The estimated projection value is used to represent the theoretical projection result formed by the current voxel occupancy probability field at each observation viewpoint.

[0061] After obtaining the estimated projection value, this embodiment compares the estimated projection value calculated by forward projection with the actual projection value of the corresponding viewpoint in the multi-angle projection dataset to calculate the projection error at each viewpoint. The projection error reflects the degree of difference between the current voxel occupancy probability field and the actual coal flow projection, and serves as an important basis for subsequent back-projection correction. A larger projection error indicates a more significant deviation between the voxel occupancy probability in the corresponding region and the actual spatial distribution of the coal flow, requiring more substantial correction in subsequent iterations.

[0062] To further improve the accuracy of 3D coal flow reconstruction, this embodiment not only utilizes traditional projection consistency constraints but also introduces depth feature information to guide voxel occupancy probabilities. Specifically, this embodiment extracts depth feature information corresponding to each viewpoint from a multi-angle projection dataset. This depth feature information can be obtained by a depth estimation network, a visual feature extraction network, or other models capable of characterizing spatial depth relationships, reflecting the local spatial structure, depth variation trend, and geometric morphology of the coal flow surface. Subsequently, this embodiment maps the depth feature information corresponding to each viewpoint to a 3D voxel space based on a unified 3D coordinate system, so that each voxel unit corresponds to a voxel feature vector that describes its spatial characteristics.

[0063] Furthermore, in this embodiment, corresponding feature guidance terms are generated based on the voxel feature vectors corresponding to each voxel unit. These feature guidance terms characterize the positional preferences of each voxel unit under depth feature constraints that better reflect the actual spatial distribution of coal flow. This ensures that the update process of voxel occupancy probability not only satisfies projection consistency but also takes into account the spatial continuity of coal flow and the consistency of local geometric structure, thereby reducing reconstruction ambiguity caused by relying solely on projection information.

[0064] Subsequently, in this embodiment, the projection error calculated from each viewpoint is used as a back-projection constraint term, and the deviation between the current occupancy probability of each voxel unit and the corresponding feature guidance term is used as a feature guidance constraint term. These two types of constraints are then combined to form a joint objective function. The back-projection constraint term ensures that the 3D reconstruction result is consistent with the actual observation results from each viewpoint, while the feature guidance constraint term ensures that the reconstruction result conforms to the actual spatial distribution characteristics of the coal flow. These two types of constraints work together to optimize the voxel occupancy probability, thereby balancing projection accuracy and spatial structure rationality.

[0065] After the joint objective function is established, this embodiment of the application uses an iterative optimization method to update the occupancy probability of each voxel unit. In each iteration, the optimization direction and update magnitude corresponding to each voxel unit are calculated according to the joint objective function, the voxel occupancy probability is corrected, and the voxel occupancy probability field for the next iteration is generated. As the number of iterations increases, the occupancy probability of each voxel unit gradually stabilizes, making the entire three-dimensional reconstruction result increasingly approximate the actual spatial distribution of coal flow.

[0066] In this embodiment, after each iteration, the change in the joint objective function between two adjacent iterations is calculated and compared with a preset convergence threshold. When the change in the joint objective function is less than or equal to the preset convergence threshold, it indicates that the current voxel occupancy probability field has reached a stable state, and continuing the iteration will have little improvement on the optimization result. Therefore, the iterative update of the occupancy probability of each voxel unit is stopped, and the converged voxel occupancy probability field is determined as the three-dimensional distribution probability field of the coal flow.

[0067] Through the iterative inversion process that combines back projection constraints with depth feature guidance, the embodiments of this application can fully integrate multi-view projection information and spatial depth feature information, effectively improve the accuracy and stability of coal flow three-dimensional spatial distribution recovery, reduce reconstruction errors caused by viewpoint occlusion, information loss and projection ambiguity in traditional voxel inversion, and provide a high-precision three-dimensional data foundation for subsequent coal flow volume calculation, coal quantity estimation and transportation status analysis.

[0068] S6. Introduce motion constraints and mass continuity constraints in the scraper conveying direction into the three-dimensional distribution probability field of coal flow, and perform spatiotemporal consistency correction on the voxel distribution at adjacent moments to obtain the dynamic three-dimensional density field of coal flow.

[0069] After obtaining the three-dimensional distribution probability field of the coal flow, this embodiment of the application further considers the temporal motion characteristics generated by the continuous conveying of the coal flow with the scraper chain. It introduces motion constraints of the scraper conveying direction and mass continuity constraints into the three-dimensional distribution probability field of the coal flow, and performs spatiotemporal consistency correction on the voxel distribution results of adjacent time points, thereby obtaining a dynamic three-dimensional density field of the coal flow that can reflect the dynamic conveying state of the coal flow.

[0070] Specifically, this embodiment first defines the three-dimensional distribution probability field of coal flow at each acquisition moment as a spatial probability distribution composed of the occupancy probabilities of multiple voxel units, where the occupancy probability of each voxel unit is used to characterize the degree to which the voxel position is occupied by the coal flow. Simultaneously, this embodiment obtains the running speed of the scraper conveyor chain at the corresponding acquisition moment and the time interval between adjacent acquisition moments. The chain running speed can be obtained from the operating parameters of the scraper conveyor drive system, speed sensor, encoder, or control system, and the time interval can be determined by the sampling frequency or timestamp information of the image acquisition unit.

[0071] Subsequently, this embodiment calculates the theoretical displacement distance of the coal flow along the scraper conveying direction between adjacent moments based on the chain running speed and the time interval between adjacent moments. This theoretical displacement distance is used to describe the amount of spatial movement that the coal flow should undergo along the scraper chain in the conveying direction under stable conveying conditions of the scraper conveyor. Further, this embodiment determines the voxel migration amount of the coal flow along the scraper conveying direction between adjacent moments based on the theoretical displacement distance and the voxel size of the three-dimensional discrete voxel mesh in the scraper conveying direction, so that the continuous motion process can be expressed in discrete voxel space.

[0072] After determining the voxel migration amount, this embodiment of the application propagates the occupancy probability of each voxel unit at the current time along the scraper conveying direction based on the voxel migration amount. Specifically, the occupancy probability of the voxel unit located upstream at the current time is migrated to the voxel unit at the corresponding position at the next time time according to the voxel migration amount, thereby obtaining the predicted occupancy probability field for the next time time. For cases where the theoretical displacement distance and voxel size do not perfectly match, this embodiment can use interpolation allocation, neighboring voxel mapping, or weighted migration methods to propagate the occupancy probability, thereby improving the continuity and accuracy of the discrete migration process.

[0073] Furthermore, this embodiment of the application uses a quality continuity constraint to compare the predicted occupancy probability field obtained from motion propagation with the three-dimensional coal flow distribution probability field obtained from multi-view inversion at the next moment on a voxel-by-voxel basis, and calculates a continuity constraint error term. This continuity constraint error term characterizes the degree of difference in voxel distribution between adjacent moments in the coal flow. When the difference between the predicted occupancy probability field and the actual three-dimensional coal flow distribution probability field obtained from inversion at the next moment is large, it indicates that there may be reconstruction errors, occlusion errors, local coal flow disturbances, or changes in transport status in the corresponding spatial region, requiring correction through spatiotemporal consistency correction.

[0074] After obtaining the continuity constraint error term, this embodiment of the application performs spatiotemporal consistency correction on the occupancy probability of each voxel unit at the next moment based on the continuity constraint error term. Specifically, during the correction process, the voxel occupancy probability at the next moment is made consistent with the predicted distribution after motion propagation at the current moment, while retaining the actual observation information obtained from the inversion of multi-angle projection data, thereby avoiding abrupt changes in the single-frame 3D reconstruction results due to dust obstruction, illumination fluctuations, local noise, or missing viewpoints. Through this correction method, the 3D distribution of coal flow at adjacent moments can satisfy the continuous motion law in the conveying direction and meet the requirements of coal flow quality continuity as a whole.

[0075] Finally, in this embodiment, the dynamic three-dimensional density field of the coal flow is calculated based on the corrected voxel occupancy probability field. Specifically, the corrected voxel occupancy probability can be correlated with the preset coal flow bulk density, coal flow particle density correction coefficient, or density estimation parameters corresponding to grayscale characteristics to obtain the density value corresponding to each voxel unit, thereby forming a dynamic three-dimensional density field of the coal flow. This dynamic three-dimensional density field of the coal flow can not only characterize the instantaneous distribution state of the coal flow in three-dimensional space, but also reflect the dynamic characteristics of the coal flow continuously changing during the scraper conveying process.

[0076] Through the spatiotemporal consistency correction process combining the above-mentioned motion constraints and mass continuity constraints, the embodiments of this application can effectively suppress random fluctuations and local anomalies in single-frame three-dimensional reconstruction results, improve the continuity, stability and physical rationality of coal flow three-dimensional distribution results in time series, and provide a more reliable dynamic three-dimensional data foundation for subsequent coal flow volume calculation, instantaneous coal quantity estimation, conveying load analysis and scraper conveyor operation status monitoring.

[0077] S7. Perform cross-sectional area integration on the three-dimensional density field of the dynamic coal flow along the conveying direction of the scraper conveyor, and calculate the volumetric flow rate of the coal flow per unit time in combination with the chain running speed.

[0078] After obtaining the dynamic three-dimensional density field of the coal flow, this embodiment of the application further calculates the volumetric flow rate of the coal flow along the conveying direction of the scraper conveyor based on the spatial distribution of the coal flow in the conveying channel, thereby realizing real-time estimation of the coal flow rate. Specifically, this embodiment of the application uses a method combining cross-sectional area integration and chain running speed to calculate the volume of coal flow passing through a specified calculation cross section per unit time, thereby obtaining the coal flow volumetric flow rate at the corresponding time.

[0079] Specifically, in this embodiment, the running direction of the scraper conveyor chain is taken as the coal flow direction, i.e., the flow rate calculation direction, in a unified three-dimensional coordinate system. Based on the spatial structure of the central trough of the scraper conveyor, at least one calculation section perpendicular to the conveying direction of the scraper conveyor is selected as the flow rate calculation section along the coal flow conveying path. This calculation section can be set in any effective observation area within the scraper conveyor channel, or multiple calculation sections can be set according to actual detection needs to achieve real-time monitoring of coal flow rate at different locations. This application does not limit this aspect.

[0080] Subsequently, in this embodiment, the dynamic coal flow three-dimensional density field is spatially discretized on the calculation cross section, dividing the calculation cross section into multiple cross-sectional voxel units to form a discrete cross-sectional region. Each cross-sectional voxel unit corresponds to one or more voxel units in the dynamic coal flow three-dimensional density field and inherits the density information of the corresponding position, thereby enabling the continuous coal flow cross section to be converted into a discrete voxel set that is convenient for numerical calculation.

[0081] Furthermore, in this embodiment, the density values ​​of each cross-sectional voxel element within the discrete cross-sectional region are calculated using cross-sectional integration to statistically analyze the spatial distribution of coal flow across the entire calculation cross-section. Specifically, the density values ​​corresponding to each cross-sectional voxel element are weighted and accumulated according to their corresponding cross-sectional areas to obtain the area-weighted density value of coal flow at the corresponding time point. This area-weighted density value comprehensively reflects the overall filling degree, spatial distribution state, and effective passage area of ​​the coal flow on the calculation cross-section. Compared to methods that only use two-dimensional projected area calculations, it can more accurately describe the true cross-sectional distribution characteristics of the coal flow.

[0082] After completing the cross-sectional integration calculation, this embodiment of the application obtains the scraper conveyor chain running speed that matches the calculated cross-section and the corresponding acquisition time. The chain running speed can be acquired in real time or read from the operating parameters output by the scraper conveyor control system, reflecting the actual conveying speed of the coal flow along the conveying direction. During normal operation of the scraper conveyor, the coal flow moves synchronously with the scraper chain; therefore, the chain running speed can be used as the coal flow speed along the conveying direction in flow rate calculation.

[0083] Subsequently, in this embodiment, the area-weighted density value of the coal flow is combined with the running speed of the scraper conveyor chain at the corresponding time to calculate the volumetric flow rate of the coal flow passing through the calculated cross section per unit time at the corresponding time. The volumetric flow rate of the coal flow can represent the total volume of the coal flow passing through the calculated cross section per unit time, and can be continuously output to form a coal flow rate time series, realizing real-time monitoring of the scraper conveyor's conveying capacity and the trend of coal flow changes.

[0084] Furthermore, the embodiments of this application can analyze the flow fluctuations, conveying load changes, and coal flow uniformity during the coal conveying process based on the coal flow volumetric flow rate calculated at multiple consecutive collection times. When abnormal fluctuations in the coal flow volumetric flow rate are detected, it can be further used as a basis for scraper conveyor operation status diagnosis, coal blockage warning, no-load identification, or conveying efficiency evaluation, thereby improving the intelligent level of scraper conveyor system operation monitoring.

[0085] Through the above calculation process, this embodiment fully utilizes the spatial distribution information contained in the three-dimensional density of dynamic coal flow, and integrates the three-dimensional reconstruction results of coal flow with the actual operating parameters of the scraper conveyor, realizing real-time, high-precision calculation of coal flow volumetric flow rate. Compared with traditional flow calculation methods that rely on single-view images or single cross-sectional area estimation, this embodiment can fully consider the three-dimensional spatial distribution characteristics and dynamic conveying laws of coal flow, effectively improving the accuracy, stability, and robustness of coal flow calculation, and providing reliable data support for coal quantity measurement, equipment scheduling, and production management in intelligent coal mine transportation systems.

[0086] S8. Calculate the mass flow rate of the coal flow based on the volumetric flow rate of the coal flow and the preset coal density parameters, and accumulate the mass flow rate over a continuous time period to obtain the cumulative coal quantity conveyed by the scraper conveyor and output it.

[0087] In this embodiment, after obtaining the volumetric flow rate of coal flow per unit time, the coal mass flow rate is further calculated by combining the coal density parameter, and the coal mass flow rate is cumulatively statistically analyzed within a continuous time range, thereby obtaining the cumulative coal conveying volume during the scraper conveyor process, realizing online measurement of coal flow and cumulative coal output.

[0088] Specifically, in this embodiment, the coal flow volumetric flow rate calculated in step S7 and the pre-set coal density parameter are used to calculate the coal flow mass flow rate at the corresponding time, thus obtaining the coal flow mass flow rate passing through the calculation section per unit time. The coal density parameter can be pre-set based on coal type, coal bulk density, on-site calibration results, or historical operating data, or it can be dynamically corrected based on changes in coal quality during actual production; this application does not limit this. By calculating the coal flow volumetric flow rate and the coal density parameter, the conversion of coal flow volume information to mass information can be achieved, enabling the coal flow delivery volume to be quantitatively expressed in mass form.

[0089] Subsequently, this embodiment of the application performs time discretization processing on the coal flow mass flow rate obtained within a continuous time range. Specifically, based on the system sampling period, image acquisition frequency, or preset statistical time interval, the continuous time is divided into multiple adjacent discrete time periods, and the coal flow mass flow rate data corresponding to each discrete time period is obtained. Through the above time discretization processing, the continuously changing coal flow mass flow rate can be formed into regular time series data, providing a unified data foundation for subsequent cumulative calculations.

[0090] Furthermore, in this embodiment, the coal flow mass flow rate corresponding to each discrete time period is cumulatively calculated to obtain the cumulative coal flow mass over a continuous time range. During the accumulation process, the system calculates the coal conveying mass within each time period sequentially based on the mass flow rate and time length corresponding to each discrete time period, and continuously accumulates them according to time order to obtain the cumulative coal conveying mass within the current statistical period. As the system continues to run, the cumulative mass value can be updated in real time and continuously reflects the total coal conveying volume of the scraper conveyor throughout the entire operation period.

[0091] After completing the cumulative calculation, this embodiment of the application associates the cumulative coal flow quality with the corresponding time information to establish a correspondence between the cumulative coal quantity and the statistical time, generating cumulative coal quantity data for the scraper conveyor. The time information may include the collection time, statistical start time, statistical end time, operating shift, production cycle, or other time information that can identify the cumulative range, facilitating subsequent data querying, historical tracing, and production statistical analysis.

[0092] Finally, this embodiment outputs the generated cumulative coal conveying data of the scraper conveyor to the monitoring terminal, control system, host computer, or mine intelligent management platform, realizing real-time display, data storage, and remote transmission of the cumulative coal conveying data. Simultaneously, this embodiment can also generate coal flow mass flow curves, cumulative coal conveying curves, and historical statistical reports based on the cumulative coal conveying data, providing reliable data support for mine production scheduling, equipment operation management, energy consumption analysis, and production statistics.

[0093] Through the aforementioned coal flow mass flow rate calculation and cumulative coal quantity statistics process, this embodiment of the application realizes a complete online metering process from three-dimensional reconstruction of the coal flow, volumetric flow rate calculation to mass measurement and cumulative statistics. Compared with traditional methods of coal quantity measurement using belt scales, nuclear scales, or manual statistics, this embodiment of the application does not require additional dedicated weighing devices. It can achieve continuous and high-precision calculation of coal flow mass flow rate and cumulative coal quantity using only multi-view visual information and scraper conveyor operating parameters. This not only improves the real-time performance and accuracy of coal quantity measurement but also enhances the system's adaptability to complex working conditions, providing reliable technical support for the automated management and digital production of intelligent transportation systems in coal mines.

[0094] Example 2 This invention also provides a deep learning-based coal conveying quantity detection system for scraper conveyors, the system comprising: The image acquisition unit is used to acquire a continuous sequence of images of coal flow from different perspectives. The preprocessing unit is used to perform time synchronization, spatial calibration and preprocessing on continuous image sequences to obtain standardized projected image data and establish a unified three-dimensional coordinate system. The projection feature extraction unit is used to extract projection contour information and grayscale distribution information from standardized projection image data to construct a multi-angle projection dataset. The voxel reconstruction unit is used to map the multi-angle projection dataset to a unified three-dimensional coordinate system and to discretize the coal flow area in the scraper conveyor channel to form a three-dimensional discrete voxel mesh. The probability inversion unit is used to iteratively invert the occupancy probability of each voxel based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel grid, using inversion constraints and depth feature guidance to obtain the three-dimensional distribution probability field of coal flow. The spatiotemporal correction unit is used to introduce motion constraints and mass continuity constraints in the scraper conveying direction into the three-dimensional distribution probability field of coal flow, and to perform spatiotemporal consistency correction on the voxel distribution at adjacent moments to obtain the dynamic three-dimensional density field of coal flow. The flow calculation unit is used to perform cross-sectional integration of the three-dimensional density field of dynamic coal flow along the conveying direction of the scraper conveyor, and calculate the volumetric flow rate of coal flow per unit time in combination with the chain running speed. The coal quantity output unit is used to calculate the coal mass flow rate based on the coal volume flow rate and the preset coal density parameters, and to accumulate the mass flow rate over a continuous time period to obtain and output the cumulative coal quantity conveyed by the scraper conveyor.

[0095] Regarding the system in the above embodiments, the specific manner in which each unit performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0096] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0097] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0098] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A deep learning-based method for detecting coal quantity in a scraper conveyor, characterized in that, Includes the following steps: Image acquisition units with different perspectives are arranged circumferentially in the scraper conveyor channel to acquire continuous image sequences of coal flow from different perspectives. Time synchronization and spatial calibration processing are performed on continuous image sequences from different perspectives to establish a unified three-dimensional coordinate system and preprocess to obtain standardized projected image data; Based on standardized projection image data, we extract the projection contour information and grayscale distribution information of coal flow from various perspectives to construct a multi-angle projection dataset. The multi-angle projection dataset is mapped to a unified three-dimensional coordinate system, and the coal flow area in the scraper conveyor channel is spatially divided by voxel discretization to form a three-dimensional discrete voxel mesh. Based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel grid, the occupancy probability of each voxel is iteratively inverted using the back projection constraint and depth feature guidance method to obtain the three-dimensional distribution probability field of coal flow. Introducing motion constraints and mass continuity constraints in the scraper conveying direction into the three-dimensional probability field of coal flow, and performing spatiotemporal consistency correction on the voxel distribution at adjacent moments, a dynamic three-dimensional density field of coal flow is obtained. The cross-sectional area of ​​the three-dimensional density field of the dynamic coal flow is integrated along the conveying direction of the scraper conveyor, and the volumetric flow rate of the coal flow per unit time is calculated in combination with the chain running speed. Based on the volumetric flow rate of the coal flow and the preset coal density parameters, the mass flow rate of the coal flow is calculated, and the mass flow rate over a continuous time period is accumulated to obtain the cumulative coal quantity conveyed by the scraper conveyor and output it.

2. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The process of obtaining a continuous image sequence includes: Based on the structural dimensions of the scraper conveyor's central trough, the coal flow trajectory, and the effective observation area, image acquisition units are installed at two locations above, to the side, and to the side of the conveying channel, forming a multi-view observation layout that has an overlapping relationship with the surface morphology and boundary contour of the coal flow. Each image acquisition unit is rigidly connected by an independent mounting frame and maintains a constant relative installation position. Based on the continuous acquisition of coal flow images by each image acquisition unit at a preset sampling frequency during the operation of the scraper conveyor, a continuous image sequence reflecting the spatial distribution of coal flow from different perspectives is obtained.

3. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The process of obtaining the standardized projection image data includes: Based on the acquisition time of continuous image sequences from different perspectives, image frames corresponding to the same transmission segment from different perspectives are time-aligned to obtain a time-synchronized multi-view image sequence. Spatial calibration is performed on each image acquisition unit to obtain the intrinsic and extrinsic parameters of each image acquisition unit; A unified three-dimensional coordinate system is established based on the external parameters of each image acquisition unit; Noise suppression processing is performed on the time-synchronized multi-view image sequence to remove image noise introduced by dust disturbance, light fluctuation and equipment vibration, resulting in a denoised image sequence; The denoised image sequence is subjected to grayscale normalization to obtain a normalized image sequence; The normalized image sequence is subjected to geometric correction processing. The distortion in the images from each viewpoint is corrected by using the intrinsic parameters of each image acquisition unit to obtain the geometrically corrected image sequence. The geometrically corrected image sequence is mapped to a unified three-dimensional coordinate system to obtain standardized projected image data.

4. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The process of constructing the multi-angle projection dataset includes: Standardized projection image data are organized according to viewpoint category and acquisition time to form a multi-view time-series image group. The coal flow region is segmented for each viewpoint image in the multi-view time-series image group to obtain a binary mask of the coal flow projection region corresponding to each viewpoint image. The outer boundary of the coal flow projection region under each viewpoint is extracted based on the binary mask of the coal flow projection region to obtain a set of boundary pixels that characterize the coal flow projection contour information. Based on the binary mask of the coal flow projection area corresponding to each viewpoint image, the pixel gray values ​​inside the coal flow projection area are read and collected to obtain the gray distribution set representing the gray distribution information of the coal flow under each viewpoint. The contour feature parameters of the coal flow projection area under each viewpoint are calculated based on the boundary pixel set, and the gray-level feature parameters of the coal flow projection area under each viewpoint are calculated based on the gray-level distribution set. By combining the set of boundary pixels, grayscale distribution, contour feature parameters, and grayscale feature parameters corresponding to each viewpoint at the same acquisition time in a unified data format, a multi-angle projection dataset corresponding to the acquisition time is constructed.

5. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The formation process of the three-dimensional discrete voxel mesh includes: Using a unified three-dimensional coordinate system as a reference, the length, width, and height ranges of the scraper conveyor channel are obtained. Based on the preset spatial resolution in the length direction, width direction, and height direction, the ranges in the length direction, width direction, and height direction are discretized to obtain the number of voxels in the length direction, width direction, and height direction of the three-dimensional discrete voxel mesh. Based on the number of voxels in the length direction, width direction, and height direction, the coal flow area in the scraper conveyor channel is spatially divided to generate voxel units. The projection contour information and grayscale distribution information of each viewpoint in the multi-angle projection dataset are mapped to a unified three-dimensional coordinate system, and the projection correspondence between image pixels and voxel units under each viewpoint is established based on the spatial position parameters and imaging direction parameters of the image acquisition units of each viewpoint. For each viewpoint, a corresponding view cone constraint region is formed on the projected contour boundary. The view cone constraint regions under each viewpoint are spatially intersected in a unified three-dimensional coordinate system to obtain the initial coal flow candidate region corresponding to the multi-angle projection dataset. Voxel elements located within the initial coal flow candidate region are assigned candidate occupancy identifiers to form a three-dimensional discrete voxel mesh corresponding to the coal flow region within the scraper conveyor channel.

6. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The process of obtaining the three-dimensional probability field of the coal flow distribution includes: Based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel mesh, the correspondence between the voxel unit and the pixel position of the projection plane under each viewpoint is established, and the occupancy probability of each voxel unit is calculated by forward projection according to the correspondence to obtain the estimated projection value. The estimated projection values ​​obtained from the forward projection calculation are compared with the actual projection values ​​in the multi-angle projection dataset to calculate the projection error at each viewpoint. The depth feature information corresponding to each viewpoint is extracted from the multi-angle projection dataset, and the depth feature information is mapped to the unified three-dimensional coordinate system to obtain the voxel feature vector corresponding to each voxel unit. Generate feature guidance terms for each voxel unit based on voxel feature vectors; The projection error is used as the back-projection constraint term, and the deviation between the occupancy probability of each voxel and the feature guidance term is used as the feature guidance constraint term. The back-projection constraint term and the feature guidance constraint term are combined to form a joint objective function. The voxel occupancy probability field in the next iteration is obtained by iteratively updating the occupancy probability of each voxel unit based on the joint objective function. When the change in the joint objective function is less than or equal to the preset convergence threshold between two adjacent iterations, the iterative update of the occupancy probability of each voxel unit is stopped, and the converged voxel occupancy probability field is determined as the three-dimensional distribution probability field of the coal flow.

7. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The process of obtaining the dynamic three-dimensional density field of coal flow includes: The three-dimensional distribution probability field of coal flow is defined as a probability distribution composed of the occupancy probability of each voxel unit, and the running speed of the scraper conveyor chain at the corresponding time and the time interval between adjacent time moments are obtained. Based on the chain running speed and time interval, calculate the theoretical displacement distance of the coal flow along the scraper conveying direction between adjacent moments; Based on the theoretical displacement distance and the voxel size of the three-dimensional discrete voxel mesh in the scraper conveying direction, the amount of voxel migration of the coal flow along the scraper conveying direction between adjacent time points is determined. Based on the voxel migration amount, the occupancy probability of each voxel unit at the current time is displaced and propagated along the scraper conveying direction, and the occupancy probability of the voxel unit at the upstream position is transferred to the voxel unit at the corresponding position at the next time, so as to obtain the predicted occupancy probability field at the next time. By using the continuity constraint of quality, the predicted occupancy probability field is compared with the probability field of the three-dimensional distribution of coal flow at the next moment on a volume-by-volume basis, and the continuity constraint error term is calculated. Based on the continuity constraint error term, the spatiotemporal consistency correction of the occupancy probability of each voxel unit in the next moment is performed to obtain the corrected voxel occupancy probability field, and the dynamic coal flow three-dimensional density field is calculated based on the corrected voxel occupancy probability field.

8. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The calculation process for the coal flow volumetric flow rate includes: In a unified three-dimensional coordinate system, the conveying direction of the scraper conveyor is taken as the flow calculation direction. A calculation section perpendicular to the conveying direction of the scraper conveyor is selected, and the three-dimensional density field of dynamic coal flow is discretized into cross-sectional voxel elements on the calculation section to form a discrete cross-sectional region. The density values ​​of each cross-sectional voxel element within the discrete cross-sectional region are calculated by cross-sectional integration to obtain the area-weighted density value of the coal flow at the corresponding time point. Obtain the scraper conveyor chain running speed that matches the calculation section and the corresponding time, and multiply the coal flow area weighted density value by the scraper conveyor chain running speed to obtain the coal flow volume flow rate per unit time through the calculation section at the corresponding time.

9. The method for detecting coal quantity in a scraper conveyor based on deep learning according to claim 1, characterized in that, The process of obtaining and outputting the cumulative coal quantity conveyed by the scraper conveyor includes: The coal flow mass flow rate at the corresponding time is calculated based on the coal flow volume flow rate and the preset coal density parameters, so as to obtain the coal flow mass flow rate passing through the calculation section per unit time. The coal flow mass flow rate within a continuous time range is discretized by dividing the continuous time into discrete time periods and obtaining the coal flow mass flow rate data corresponding to each discrete time period. The cumulative mass flow rate of coal in each discrete time period is calculated to obtain the cumulative mass of coal in the continuous time range. The cumulative coal flow quality is correlated with the corresponding time information to generate and output the cumulative coal quantity data of the scraper conveyor.

10. A deep learning-based scraper conveyor coal quantity detection system, comprising the deep learning-based scraper conveyor coal quantity detection method according to any one of claims 1 to 9, characterized in that, include: The image acquisition unit is used to acquire a continuous sequence of images of coal flow from different perspectives. The preprocessing unit is used to perform time synchronization, spatial calibration and preprocessing on continuous image sequences to obtain standardized projected image data and establish a unified three-dimensional coordinate system. The projection feature extraction unit is used to extract projection contour information and grayscale distribution information from standardized projection image data to construct a multi-angle projection dataset. The voxel reconstruction unit is used to map the multi-angle projection dataset to a unified three-dimensional coordinate system and to discretize the coal flow area in the scraper conveyor channel to form a three-dimensional discrete voxel mesh. The probability inversion unit is used to iteratively invert the occupancy probability of each voxel based on the projection relationship between the multi-angle projection dataset and the three-dimensional discrete voxel grid, using inversion constraints and depth feature guidance to obtain the three-dimensional distribution probability field of coal flow. The spatiotemporal correction unit is used to introduce motion constraints and mass continuity constraints in the scraper conveying direction into the three-dimensional distribution probability field of coal flow, and to perform spatiotemporal consistency correction on the voxel distribution at adjacent moments to obtain the dynamic three-dimensional density field of coal flow. The flow calculation unit is used to perform cross-sectional integration of the three-dimensional density field of dynamic coal flow along the conveying direction of the scraper conveyor, and calculate the volumetric flow rate of coal flow per unit time in combination with the chain running speed. The coal quantity output unit is used to calculate the coal mass flow rate based on the coal volume flow rate and the preset coal density parameters, and to accumulate the mass flow rate over a continuous time period to obtain and output the cumulative coal quantity conveyed by the scraper conveyor.

Images