Stone coal particle size detection and separation control method based on machine vision deep learning

Abstract

The application discloses a stone coal particle size detection and separation control method based on machine vision deep learning, and relates to the technical field of detection, wherein the method comprises the following steps: an electrostatic sensor is used for monitoring the material flow in a pipeline, and a high-speed camera and a stroboscopic light source are synchronously triggered to shoot particle images; after image dust removal and enhancement, the image is sent to a lightweight segmentation model, and a single particle contour is accurately segmented; the equivalent particle diameter of each particle is calculated, the particle size distribution is counted, and a soft measurement model is used for calculating the carbon content of the material; combined with the two data, a separation adjustment instruction is output by a self-adaptive threshold controller, and an execution mechanism is driven to adjust the separation working condition in real time. The application realizes continuous online real-time detection and intelligent separation closed-loop control of the particle size of stone coal particles, significantly improves the coal dust recovery rate, and reduces the equipment wear.

Description

Technical Field

[0001] This invention relates to the field of detection technology, and in particular to a method for controlling the detection, sorting and control of stone and coal particle size based on machine vision deep learning. Background Technology

[0002] As a waste product discharged from coal mills in coal-fired power plants, the efficient treatment and resource utilization of stone coal is an important link in ensuring the continuous operation of power plants and energy conservation and emission reduction.

[0003] However, existing stone coal discharge systems directly use offline screening or static image analysis for particle size detection. They do not have a real-time detection solution designed for the complex working conditions of high-speed pneumatic conveying pipelines with high material flow rates (18-30 m / s), uneven particle distribution, and overlapping and adhesion. This may lead to significant randomness and lag in the detection results, making it impossible to track the changing trend of material flow. Consequently, it affects the accuracy of effective recovery and sorting control of coal powder particles, resulting in the waste of combustible resources in stone coal and exacerbating the wear of coal mill equipment. Summary of the Invention

[0004] The main objective of this invention is to provide a method for controlling the detection, sorting, and separation of stone and coal particles based on machine vision deep learning.

[0005] Another objective of this invention is to propose a stone and coal particle size detection and sorting control device based on machine vision deep learning.

[0006] The third objective of this invention is to provide an electronic device.

[0007] A fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0008] To achieve the above objectives, a first aspect of the present invention proposes a method for controlling the particle size detection and sorting of gravel and coal based on machine vision deep learning, comprising:

[0009] S1 detects the flow state of the material in the pipeline through an electrostatic trigger sensor, generates a trigger signal to control the synchronization of the high-speed camera and the high-frequency stroboscopic light source, and acquires dynamic particle images in the conveying pipeline. S2, perform dust removal and enhancement preprocessing on the acquired image to remove coal dust occlusion and enhance particle edge contours. Input the preprocessed image into the lightweight instance segmentation model, perform pixel-level segmentation of particles through adaptive high-resolution branch, and output the mask and bounding box of each particle. S3, based on the segmentation results, calculate the equivalent particle size of each particle, and perform particle size distribution statistics according to the preset particle size range. At the same time, combine the carbon content soft measurement model to estimate the carbon content of the material. S4, based on the particle size distribution statistics and carbon content estimation, generates sorting parameter adjustment instructions through an adaptive threshold controller, and performs sorting control through an actuator to obtain real-time corrected controller parameters.

[0010] Optionally, an electrostatic trigger sensor detects the flow state of the material inside the pipeline, generates a trigger signal to control the synchronization of a high-speed camera and a high-frequency stroboscopic light source, and acquires dynamic particle images inside the conveying pipeline, including: The non-contact ring electrode electrostatic sensor installed upstream of the observation section is used to detect the charged state of the material in the pipeline in real time. When the material is detected to have reached the sensor detection area, a TTL pulse signal is output to the external trigger input terminal of the camera controller within 10 microseconds. Based on the real-time flow rate of the material and the distance between the sensor and the observation section, the trigger delay compensation amount is calculated through the flow rate-delay correspondence table to generate a timing-calibrated trigger signal. The trigger signal is sent to a high-frequency LED strobe light source and a high-speed industrial camera, and the pulse width of the light source is controlled to synchronize with the camera exposure time so that the camera completes the exposure when the particle is in the optimal imaging position, and acquires dynamic particle images in the conveying pipeline.

[0011] Optionally, the acquired image undergoes dust removal and enhancement preprocessing to remove coal dust occlusion and enhance particle edge contours. The preprocessed image is then input into a lightweight instance segmentation model, which performs pixel-level segmentation of the particles through an adaptive high-resolution branch, outputting the mask and bounding box for each particle, including: The dark channel map is calculated for the acquired image. The average value of the original image pixels corresponding to the top 0.1% of the brightest pixels in the dark channel map is taken as the atmospheric light value. The transmittance map is estimated and a soft matting algorithm is used to refine the transmittance map to complete the dust removal image restoration. The dust-removed grayscale image is divided into 8×8 non-overlapping sub-blocks. A histogram is calculated for each sub-block. The contrast limit threshold is set to 4 times the number of gray levels. Histogram portions exceeding the threshold are truncated and redistributed. Histogram equalization is performed on each sub-block. Bilinear interpolation is used to eliminate discontinuities at the sub-block boundaries to obtain the enhanced image. The system identifies and removes blank frames without material, overexposed or underexposed invalid frames, and inputs the valid images into the lightweight instance segmentation model.

[0012] Optionally, the lightweight instance segmentation model uses YOLOv8-seg as the backbone network. The YOLOv8-seg network consists of three parts: a backbone network, a neck network, and a segmentation head. The backbone network adopts a CSPDarknet structure containing 5 convolutional layers and 4 C2f modules. The neck network adopts a PANet structure, which enhances multi-scale object detection capabilities through top-down and bottom-up feature fusion. The segmentation head includes a detection branch and a segmentation branch that output the target bounding box, category, and pixel-level mask, respectively. An adaptive high-resolution branch is added after the last C2f module of the backbone network. This branch contains one 2x upsampling layer, one 3×3 convolutional layer, and one feature fusion module.

[0013] Optionally, based on the segmentation results, the equivalent particle size is calculated for each particle, and particle size distribution statistics are performed according to a preset particle size range. Simultaneously, the carbon content of the material is estimated using a soft-sensing model, including: The camera intrinsic and extrinsic parameters and distortion correction were completed using a checkerboard calibration board via binocular vision calibration, and the pixel equivalent was determined. Extract the boundary pixel coordinates of each particle mask, fit the ellipse parameters using the direct least squares method, calculate the major axis a, minor axis b, and rotation angle θ, and calculate the equivalent particle size using the geometric mean. Simultaneous calculation of aspect ratio As an auxiliary parameter for particle morphology; Histogram statistics are performed in five ranges: 0-1mm, 1-3mm, 3-6mm, 6-10mm, and greater than 10mm. The statistical period is adaptively set to 100ms-1s, and the P10, P50, and P90 particle size values ​​and powder content are calculated simultaneously.

[0014] Optionally, based on the particle size distribution statistics and carbon content estimation, an adaptive threshold controller generates sorting parameter adjustment commands, which are then used to control the sorting process via an actuator, resulting in real-time corrected controller parameters, including: The powder content, P50 particle size value, and carbon content estimate are input into the adaptive threshold controller. The controller is implemented using fuzzy inference. First, the powder content, P50, and carbon content are divided into three fuzzy subsets: low, medium, and high, and a triangular membership function is used. The size of the sorting plate hole is divided into five fuzzy subsets: very small, small, medium, large, and very large, and a trapezoidal membership function is used. 25 complete fuzzy rules are formulated. The centroid method is used to complete the defuzzification and output the precise control values ​​of the sorting plate hole size and the pneumatic conveying speed. The control system transmits data to the adjustable aperture sorting plate and the variable frequency fan actuator via the TCP industrial bus, synchronously collecting particle size data of the sorted material as feedback, and correcting the controller parameters in real time. The entire closed-loop control cycle does not exceed 33ms.

[0015] Optionally, during the model training phase, a conditional generative adversarial network is used to augment the data of wet particle adhesion and overlap scenarios. The generator takes a single particle image and particle size label as input and outputs a synthetic sample of adhered or overlapping particles. The discriminator distinguishes between real samples and synthetic samples. The generation effect is optimized through adversarial training. During training, real samples and synthetic samples are mixed in a 7:3 ratio to expand the diversity of the dataset.

[0016] To achieve the above objectives, a second aspect of the present invention provides a stone and coal particle size detection and sorting control device based on machine vision deep learning, comprising: The acquisition unit is used to detect the flow state of the material in the pipeline through an electrostatic trigger sensor, generate a trigger signal to control the synchronization of the high-speed camera and the high-frequency stroboscopic light source, and acquire dynamic particle images in the conveying pipeline. The preprocessing unit is used to perform dust removal and enhancement preprocessing on the acquired image, remove coal dust occlusion and enhance particle edge contours. The preprocessed image is input into the lightweight instance segmentation model, which performs pixel-level segmentation of particles through an adaptive high-resolution branch and outputs the mask and bounding box of each particle. The calculation unit is used to calculate the equivalent particle size of each particle based on the segmentation results, perform particle size distribution statistics according to the preset particle size range, and estimate the carbon content of the material by combining the soft measurement model of carbon content. The sorting control unit is used to generate sorting parameter adjustment instructions through an adaptive threshold controller based on the particle size distribution statistics and carbon content estimation, and to perform sorting control through an actuator to obtain real-time corrected controller parameters.

[0017] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0018] To achieve the above objectives, a third aspect of this application provides an electronic device, including a processor and a memory; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, for implementing the stone and coal particle size detection and sorting control method based on machine vision deep learning as described in the first aspect embodiment.

[0019] To achieve the above objectives, the fourth aspect of this application proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the stone and coal particle size detection and sorting control method based on machine vision deep learning as described in the first aspect embodiment.

[0020] The embodiments of the present invention have the following beneficial effects: Through adaptive sorting control, the system can automatically adjust the sorting parameters based on the real-time detected particle size distribution and carbon content, ensuring that the sorting effect is always optimal and improving the coal powder recovery rate. It achieves effective separation and recovery of coal powder particles from gravel coal, avoiding the wear of grinding rollers, grinding discs and other equipment caused by the re-entry of high-hardness gravel coal particles into the furnace, and is expected to extend the maintenance cycle of the coal mill by more than 30%. The entire detection and sorting process is fully automated, requiring no manual intervention, reducing the workload of operators, and also reducing quality fluctuations caused by human error. Attached Figure Description

[0021] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart of a stone and coal particle size detection and sorting control method based on machine vision deep learning provided in an embodiment of the present invention; Figure 2 This is a structural diagram of a stone and coal particle size detection and sorting control device based on machine vision deep learning, provided in an embodiment of the present invention. Detailed Implementation

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] The following description, with reference to the accompanying drawings, describes a method and apparatus for detecting and sorting stone and coal particle size based on machine vision deep learning, according to embodiments of the present invention.

[0025] Example 1 This invention provides a method for controlling the detection, sorting, and separation of stone and coal particles based on machine vision deep learning, such as... Figure 1 As shown, the method includes the following steps: S1 detects the flow state of the material in the pipeline through an electrostatic trigger sensor, generates a trigger signal to control the synchronization of the high-speed camera and the high-frequency stroboscopic light source, and acquires dynamic particle images in the conveying pipeline.

[0026] To address the issues of particle imaging ghosting, misaligned acquisition timing, and excessive invalid frames caused by high dust and high flow velocity in power plant coal and stone pipelines, this application utilizes electrostatic timing-triggered linkage light source cameras, along with optimized observation hardware, to accurately acquire high-quality dynamic particle images.

[0027] In this application embodiment, existing online visual inspection equipment for power plant stone and coal generally suffers from several intractable industrial pain points: high-speed dust inside the pipeline causes large-area fogging and obstruction of the image; the conveying velocity of stone and coal particles can reach up to 30m / s, which easily leads to motion blur; and the timing of material arrival at the imaging area cannot be accurately synchronized, resulting in a large number of blank and invalid frames, which significantly reduces the accuracy of subsequent particle size detection and the efficiency of system operation. To address the above deficiencies, this application specifically deploys an integrated imaging hardware unit in the transparent observation section of the pipeline. High-frequency LED strobe light sources and high-speed industrial cameras are respectively installed on both sides of the transparent observation section of the stone and coal pneumatic conveying pipeline, and electrostatic trigger sensors and local image preprocessing units are deployed in tandem to achieve time-series linkage and synchronous imaging of multiple imaging hardware, solving the problems of blurred high-speed particle imaging and misaligned acquisition timing from the hardware level.

[0028] In this embodiment, the transparent observation section of the imaging core carrier is specially designed with optimized structure and materials. The entire transparent observation section is made of high-strength wear-resistant quartz glass in one piece, with a glass thickness of not less than 15mm, a light transmittance of not less than 92% in the visible light band, and a fixed refractive index of 1.45. Its mechanical properties can stably withstand the -0.06MPa negative pressure load inside the pipeline and the continuous impact and scouring of high-speed stone and coal particles. The effective observation length of the observation section is uniformly designed to be 300mm. The inner diameter of the observation section is completely consistent with the inner diameter of the material conveying pipeline before and after it, which can ensure that the high-speed camera can completely cover the particle flow area of ​​the entire cross section of the pipeline without any blind spots.

[0029] In this embodiment, the left and right ends of the observation section are sealed to the metal conveying pipeline via standard flange structures. Food-grade high-elasticity silicone sealing rings are installed at the flange connection points to effectively prevent the internal airflow of the pipeline from leaking outward and the coal dust from overflowing inward and contaminating the imaging area. At the same time, the inner wall of the observation section is integrally formed with a streamlined flow guiding structure, which can suppress the disordered turbulence generated when the high-speed airflow passes through the observation section, effectively avoiding the problem of particles adhering to the wall and uneven spatial distribution of particles due to the influence of airflow. The outer surface of the observation section is equipped with a detachable dust cover and an acrylic anti-collision protective cover to isolate the floating coal dust at the power plant site from adhering to the glass outer wall, while resisting the external force collision caused by on-site maintenance tools and materials splashing, extending the service life of the imaging observation section and reducing the frequency of on-site maintenance.

[0030] In this embodiment, a high-frequency LED strobe light source is fixedly installed on the outer wall of one side of the pipe at the corresponding observation area position. The light emission center of the light source is strictly aligned with the central axis of the pipe, the beam divergence angle is fixed at 60 degrees, and the light source body and the high-speed industrial camera lens are arranged at a 90-degree angle to each other. This can completely avoid imaging interference problems such as mirror reflection and image overexposure caused by the light source shining directly into the lens.

[0031] In this embodiment, the hardware performance parameters of the stroboscopic light source have been optimized for operating conditions, with a maximum pulse output frequency of not less than 10kHz, a pulse rise and fall time of less than 1 microsecond, a peak luminous power of 50W, and a pulse width supporting 1... Up to 5 Continuously adjustable, the camera exposure time can be matched according to the material flow rate to eliminate the ghosting of high-speed particles; the high-speed industrial camera used in conjunction with it is equipped with a global shutter CMOS image sensor with an effective imaging pixel of no less than 2 million and an output resolution of 1920×1080. The camera lens adopts a dual-sided telecentric lens, which can completely eliminate the interference of imaging perspective distortion on particle size measurement accuracy. The overall imaging resolution of the system is no less than 0.1mm per pixel, which can clearly identify ultrafine stone and coal particles with a minimum size of 0.5mm.

[0032] In this embodiment, the high-speed camera is fixedly mounted on a three-dimensional adjustable precision metal bracket, allowing on-site personnel to perform multi-dimensional fine adjustments in up / down, forward / backward, and left / right directions, facilitating on-site equipment calibration and lens focusing. The actual shooting frame rate of the camera can be adaptively and dynamically adjusted according to the real-time flow rate of the material inside the pipeline. Constraints are designed based on the maximum on-site material conveying speed of 30 m / s to ensure that the displacement of a 10 mm diameter coarse particle within two adjacent frames does not exceed one-third of its own diameter. Based on this, the minimum operating frame rate of the camera is set to be no less than 1000 fps. Under different loads and flow rates, the system can automatically and adaptively adjust the camera's shooting frame rate within the range of 500 fps to 5000 fps, balancing image clarity and real-time data processing.

[0033] In this embodiment, a non-contact ring electrode electrostatic sensor is used as the material arrival detection device. The electrostatic sensor is installed on the outer wall of the pipe 500mm upstream of the transparent observation section. The sensor is completely covered with a metal electromagnetic shield, which can isolate industrial electromagnetic interference generated by power plant frequency converters, fans and other equipment, and ensure stable electrostatic signal acquisition.

[0034] In this embodiment, the electrostatic sensor signal detection bandwidth is 10kHz, and the detection sensitivity supports graded adjustment. It can stably identify charged stone and coal particles with a minimum diameter of 0.5mm. When material particles continuously enter the sensor detection area, the electrostatic induction voltage on the outer wall of the pipe will undergo a significant change. After the built-in trigger signal generation circuit captures the voltage change, it can quickly output a standard TTL pulse signal within 10 microseconds and transmit it to the external trigger input terminal of the camera controller to complete the hardware synchronous transmission of the material arrival signal.

[0035] In this embodiment, to further eliminate the acquisition misalignment problem caused by material transport delay between the sensor and the imaging section, a dedicated flow velocity timing calibration mechanism is added. The system reads the instantaneous flow velocity of the material in the pipeline in real time, and, combined with the fixed distance between the sensor installation position and the center of the transparent observation section, retrieves the locally stored flow velocity-delay lookup table to accurately calculate the trigger delay compensation amount. Based on the compensation amount, a timing-calibrated synchronous trigger signal is generated. After the calibration signal is output, the system synchronously controls the pulse width of the high-frequency LED strobe light source and the exposure time of the high-speed industrial camera, strictly matching the timing to ensure that the camera completes synchronous exposure acquisition when the particulate material just moves to the center of the imaging area. The total trigger delay of the entire link, including signal detection, delay compensation, and synchronous output of the light source and camera, does not exceed 10 microseconds. This can significantly reduce invalid images such as blank frames without material and misaligned blurred frames, improve image acquisition efficiency, and ultimately stably and continuously acquire dynamic images of high-speed flowing stone and coal particles inside the pneumatic conveying pipeline.

[0036] In this embodiment, by optimizing the structure of the quartz glass observation section, deploying angled imaging hardware, and implementing electrostatic sensing delay calibration linkage control, precise synchronization of material timing is achieved, and low-latency acquisition of dynamic particle images of stones and coal without ghosting and with high efficiency is realized, laying the foundation for subsequent visual inspection.

[0037] S2 performs dust removal and enhancement preprocessing on the acquired image to remove coal dust occlusion and enhance particle edge contours. The preprocessed image is then input into the lightweight instance segmentation model, which performs pixel-level segmentation of the particles through an adaptive high-resolution branch, and outputs the mask and bounding box of each particle.

[0038] To overcome the shortcomings of pipeline coal dust interference, uneven segmentation accuracy of coarse and fine particles, and poor segmentation effect under complex working conditions, this application adopts a tandem image preprocessing algorithm, improves the YOLOv8-seg network and combines it with CGAN data augmentation to accurately complete pixel-level segmentation of particles.

[0039] In this embodiment, the power plant's stone and gas transmission pipeline is in a long-term state of high coal dust and high water vapor mixture. The diffused coal dust will directly cause the acquired image to be hazy and degraded, the particle edges to be blurred and stuck, and the particle texture to be missing. Conventional single image enhancement algorithms cannot take into account both the dust removal effect and the integrity of the particle edges, which can easily cause small particles to be dissolved and large particles to have distorted contours. To address the above-mentioned pain points in the industry's imaging processing, this application uses a series coupled dark channel prior dust removal algorithm and contrast-limited adaptive histogram equalization (CLAHE) algorithm to complete image noise reduction and contour enhancement processing in steps, taking into account both the dust removal and purification effect and the particle edge preservation effect.

[0040] In this embodiment, the dark channel prior dehazing algorithm performs image dehazing and dehazing processing according to a standardized five-step operation flow. The first step is to calculate the dark channel image of the input RGB color image, and the calculation formula is as follows:

[0041] in, This refers to the dark channel values ​​of an RGB color image. The algorithm uses a fixed 15×15 local computation window with x and y coordinates. For color channel set, The first step is to obtain a single-channel grayscale image corresponding to the RGB three channels; the second step is to refine the image by selecting the average pixel value of the original image corresponding to the top 0.1% of the pixels with the highest brightness in the dark channel image, and accurately calibrate this average value as the ambient atmospheric light value A, thereby improving the atmospheric light value to better match the on-site dust lighting environment; the third step is to estimate the initial transmittance map of the image by combining the on-site dust scattering coefficient, using the following formula:

[0042] in, The initial transmittance of the image. Here, A is the adjustment coefficient, and A is the atmospheric light value.

[0043] The fourth step introduces a soft matting algorithm to iteratively optimize and refine the global transmittance map, eliminating blocky tortuosities and edge artifacts present in the basic transmittance map; the fifth step is based on the image restoration formula:

[0044] in, For the original clear image, The original, murky image was captured. This is used to avoid the denominator being zero and to ensure the stability of the algorithm.

[0045] In this embodiment, after coal dust removal, the CLAHE enhancement algorithm performs edge enhancement processing on the clean image. The algorithm first converts the color dust-removed image into a single-channel grayscale image, divides the entire grayscale image into several 8×8 non-overlapping sub-blocks, independently calculates the grayscale distribution histogram for each sub-block, sets the contrast limit threshold to four times the number of grayscale levels in the image, automatically truncates the grayscale histogram amplitude exceeding the threshold and rebalances the grayscale distribution, optimizes the local brightness and darkness contrast, and finally uses a bilinear interpolation algorithm to smooth out the color difference and discontinuity problems at the splicing boundaries of adjacent sub-blocks, comprehensively enhancing the recognizability of the edge contours of stone and coal particles and distinguishing the textures at the boundaries of adhering particles.

[0046] In this embodiment, after the image enhancement process is completed, the system is equipped with an intelligent frame discrimination algorithm to automatically identify and remove three types of invalid imaging frames: blank frames with no material flowing in the pipeline, overexposed imaging frames with lost particle information due to overexposure of the light source, and underexposed imaging frames with insufficient light source illumination and missing details. Only valid particle images with complete particle information and qualified imaging quality are selected and retained, and sent to the backend instance segmentation model to reduce the consumption of invalid computing power.

[0047] In this embodiment, the existing basic YOLOv8-seg model has a fixed computational cost and cannot adapt to the differentiated segmentation requirements of coarse and fine mixed stone and coal particles. The segmentation contour accuracy of large particles is insufficient, and small particles are easily missed. This application selects YOLOv8-seg as the basic backbone network. The network architecture is standardized and divided into three major functional modules: backbone network, neck network, and segmentation head. The backbone network adopts a lightweight CSPDarknet feature extraction structure, with 5 standard convolutional layers and 4 C2f feature fusion modules built in. It extracts multi-scale particle features of shallow texture and deep semantics of the image layer by layer, which can adapt to the feature extraction of particles of different sizes. The neck network adopts a PANet bidirectional feature fusion structure, which links the dual logic of top-down feature upsampling fusion and bottom-up feature downsampling fusion to fuse deep and shallow features, and simultaneously take into account the detailed features of ultra-fine particles and the overall features of ultra-large particles. The segmentation head is coupled with independent detection branches and independent segmentation branches in parallel, and can output three types of structured data in parallel: bounding boxes of stone and coal particles, particle size categories, and pixel-level full-coverage masks.

[0048] In this embodiment, the application addresses the issues of imbalanced precision in coarse and fine particle segmentation and wasted computational power due to model idleness by making core architectural improvements. An adaptive high-resolution branch is specifically added after the last C2f feature extraction module in the backbone network. This branch's hardware structure is fixed, consisting of a set of 2x upsampling layers, a set of 3×3 fine convolutional layers, and a set of channel feature fusion modules. The branch incorporates a controllable hard-gated activation mechanism. A lightweight, dedicated particle size discriminator is added after the segmentation head. This discriminator consists of a global average pooling layer, a single fully connected layer, and a Sigmoid nonlinear activation function. The system is structured with a uniform branch activation threshold of 0.5. When the discriminator outputs a value greater than 0.5, indicating the presence of coarse particles larger than 6mm in the image, the high-resolution branch is immediately activated. This branch integrates refined high-resolution features with the main basic features through channel stitching, filling in the edge details of large particles and optimizing the contour accuracy of coarse particle segmentation. When the discriminator outputs a value less than or equal to 0.5, indicating the presence of only small to medium-sized particles in the image, the high-resolution branch automatically goes into hibernation, and its feature output is set to zero. This eliminates the need for feature computation, effectively reducing the model's idle computing power loss and adapting to low-computing-power operating conditions on edge hardware.

[0049] In this embodiment, the model uses a standardized input of a 640×640 three-channel color image. After lightweight architecture optimization, the model's single-frame inference speed is no less than 30 FPS, and the single-frame image processing latency is no more than 33 ms, which is fully compatible with the millisecond-level real-time processing deployment requirements of power plant industrial edge control terminals.

[0050] In this embodiment, considering the high-frequency and complex working conditions of wet material adhesion and particle stacking at power plant sites, and the fact that conventional datasets have simple working conditions and poor model generalization ability, this application introduces a conditional generative adversarial network (CGAN) to perform working condition-specific data augmentation during the offline training stage of the model. The CGAN generator takes real single-particle images collected on site and accurate particle size labels as bidirectional inputs to simulate and generate synthetic samples of wet adhesion and stacking of stones and coal that fit the working conditions on site. The discriminator autonomously distinguishes between real samples on site and synthetic samples synthesized by the algorithm. Through bidirectional adversarial iterative training, the realism of the synthetic sample imaging is continuously optimized.

[0051] In this embodiment, during the offline training phase of the model, real samples captured on-site and CGAN-synthesized samples are mixed in a 7:3 ratio to construct a full-domain training dataset. This enriches the number of samples in complex working conditions such as adhesion, overlap, and dampness, thereby improving the model's anti-interference segmentation and generalization capabilities in complex power plant environments. After the offline training of the model converges, the model is quantized and deployed according to the TensorRT high-performance inference framework, which can stably and reliably operate on power plant industrial edge computers and embedded low-end control hardware platforms.

[0052] In this embodiment, the dark channel combined with the CLAHE algorithm is used to purify and enhance the image and remove invalid images. An adaptive high-resolution branch is added to optimize the model structure. Based on the CGAN expanded dataset, accurate particle masks and bounding boxes are efficiently output, laying the foundation for subsequent particle size calculation.

[0053] S3 calculates the equivalent particle size for each particle based on the segmentation results, performs particle size distribution statistics according to the preset particle size range, and estimates the carbon content of the material by combining the soft measurement model of carbon content.

[0054] In order to accurately obtain the physical properties, particle size distribution data, and real-time carbon content of the stone coal particles to support the back-end sorting and control operations, this application combines the bi-objective fixed ellipse fitting calculation parameters, classifies and statistically analyzes the particle size index, and estimates the carbon content based on the composite soft measurement model.

[0055] In this embodiment, single-vision cameras are prone to lens radial distortion and installation angle deviation, leading to particle size measurement errors that cannot meet the industrial measurement accuracy requirements for power plant stone and coal particle size detection. This application specifically adopts a binocular vision calibration method to complete the parameter calibration and correction of the entire imaging system. On-site, calibration images are collected from multiple points according to the national standard 20×20 checkerboard calibration board. The three core parameters of high-speed camera built-in intrinsic parameters, on-site installation extrinsic parameters, and lens optical distortion are iteratively calibrated across the entire domain. The real-time pixel equivalent is accurately calibrated on-site, and the physical millimeter size corresponding to a single pixel is clearly defined. After the entire calibration process is completed, the particle size measurement calibration error in the entire scene is strictly controlled within 0.5%, meeting the industrial online particle size detection measurement standards and ensuring compliance and accuracy of subsequent particle size calculations throughout the entire process.

[0056] In this embodiment, the system calls the image processing interface to extract the sub-pixel coordinates of the single-particle mask contour edge, improving the edge coordinate sampling accuracy. The Fitzgibbon direct least squares method, adapted for fitting irregular coal particles, is used to fit the circumscribed equivalent ellipse of the particle, efficiently and stably solving for the ellipse's major axis a, minor axis b, and plane rotation angle. Three geometric parameters, converted using the geometric mean equivalent formula Accurate solution of equivalent particle size It is adapted to the particle size measurement requirements of non-spherical and irregularly shaped coal particles and simultaneously calculates the particle aspect ratio. Based on the numerical quantification of aspect ratio, the morphological characteristics of particles, such as flatness and regularity, are characterized, which helps to improve the dimensions of material property data analysis.

[0057] In this embodiment, combining the power plant's stone and coal sorting process standards and on-site screening process requirements, this application pre-sets five fixed particle size grading intervals. These intervals align with power plant operation and maintenance control indicators, namely 0-1mm ultrafine particles, 1-3mm fine particles, 3-6mm medium particles, 6-10mm coarse particles, and larger than 10mm ultralarge particles. The system can adaptively switch between an adjustable statistical period of 100ms to 1s based on pipeline material transport load and particle size fluctuation amplitude, generating a real-time particle size distribution histogram and simultaneously calculating industry-standard P10, P50, and P90 particle size characteristic percentiles. The general formula for percentile calculation is:

[0058] in, This is the particle size value. The target percentile is set as N, the total number of valid particles in the current statistical period is N, and d is the particle size array after sorting in ascending order from smallest to largest.

[0059] In this embodiment, the system calculates the core powder content index of the process in real time. This application defines the powder content as the percentage of the total mass of fine particles with a diameter less than 3mm within a single cycle, relative to the total mass of all materials. The corresponding calculation formula is:

[0060] in, For flour content, The equivalent particle size of a single particle. For the mass of a single particle, an ellipsoidal volume model adapted to the coal morphology is used for calculation. The calculation formula is as follows: In the formula To adaptively identify density parameters for materials, the system has built-in adaptive density matching logic. The density of raw coal powder under loose working conditions is fixed at 1.5 g / cm³, and the density of hard stone coal under working conditions is fixed at 2.4 g / cm³. The system automatically determines the material type based on the average gray value of the particle surface and the length and width characteristics of the particle morphology, and autonomously matches the corresponding density parameters without the need for manual on-site assignment and debugging.

[0061] In this embodiment of the application, in order to avoid interference from distorted particles and oversized abnormal particles in the statistical results, the system has built-in exclusive abnormal particle removal rules. It automatically identifies and removes invalid particles that are too sticky to complete ellipse fitting, whose particle size exceeds the detection range of the equipment, or whose imaging is incomplete. Such particle data are not included in the particle size and quality full-domain statistical process, ensuring that the particle size distribution and powder content calculation results are accurate and reliable from the data source.

[0062] In this embodiment, traditional carbon content detection relies on offline sampling and testing, which has a strong detection lag and cannot adapt to real-time sorting conditions. This application builds a composite soft measurement model for carbon content that integrates multivariate linear regression and BP neural network, avoiding the cost of installing professional testing hardware. The model selects multidimensional features such as the current material powder content, P50 median particle size, and particle proportion in five ranges as input variables. Iterative training and optimization are completed based on the offline sampling and testing calibration dataset of stone and coal from power plants over the years. After optimization, the real-time prediction error of carbon content in stone and coal materials is stable and does not exceed 5%. It can estimate the carbon content of materials flowing inside pipelines without contact and with low latency, providing reliable physical property parameters for back-end sorting closed-loop control.

[0063] In this embodiment, the measurement error is strictly controlled by binocular vision calibration, the particle parameters are solved by least squares fitting, the particle size and powder content are adaptively classified by statistical method, invalid particle data are eliminated, and the carbon content of the material is calculated online by combining a trained neural network model, so as to provide accurate working condition data for adaptive sorting control.

[0064] S4, based on the particle size distribution statistics and carbon content estimation, generates sorting parameter adjustment instructions through an adaptive threshold controller, and performs sorting control through an actuator to obtain real-time corrected controller parameters.

[0065] In order to adapt to the dynamic fluctuations of coal and stone working conditions in power plants and to take into account industrial control equipment with different computing power, and to achieve accurate, stable and intelligent sorting, this application sets up a dual-mode adaptive controller, which issues instructions in combination with material property parameters, and dynamically corrects parameters based on feedback to build a millisecond-level sorting closed-loop system.

[0066] In this embodiment, the computing power levels of power plant field control hardware vary significantly. High-end industrial control computers have sufficient computing power, while embedded edge controllers have limited computing power. The single control mode has poor adaptability and is prone to problems such as control lag and poor sorting adaptability. The adaptive threshold controller of this application has a built-in switchable dual operating mode, namely fuzzy inference operating mode and lightweight neural network operating mode. The two modes can be switched with one click according to the computing power of the field hardware, which fully adapts to various industrial hardware platforms with high and low computing power.

[0067] In this embodiment, the fuzzy inference operation mode is adapted to the mainstream industrial control hardware and complex and variable sorting conditions in power plant sites. It has the advantages of strong control stability and strong resistance to operating condition disturbances. This mode selects three coupled physical property parameters, namely powder content, median P50 particle size, and carbon content of materials, as input variables of the controller, and uses the adjustable sorting plate hole opening size and variable frequency fan pneumatic conveying speed as the terminal execution output control quantity to realize the linkage control from physical property parameters to sorting execution parameters.

[0068] In this embodiment, the application completes the standardization and fuzzification assignment of fuzzy control variables across the entire domain. The three input variables—powder content, P50 particle size, and carbon content—are uniformly divided into three levels of fuzzy subsets: low, medium, and high. All subsets are matched with triangular membership functions to complete membership degree calculations. The core execution variable, the sorting plate hole size, is divided into five levels of fuzzy subsets: very small, small, medium, large, and very large. Trapezoidal membership functions are matched to improve the smoothness of boundary control. Combined with the coal and stone sorting process in power plants, 25 full-coverage, full-condition linkage fuzzy control rules are established to supplement the control logic for extreme conditions. The core control rules include reducing the sorting plate hole size under high powder content and high carbon content conditions, increasing the sorting plate hole size under low powder content and low carbon content conditions, and increasing the variable frequency fan pneumatic conveying speed under conditions where the P50 particle size is generally coarser. After all fuzzy inference calculations are completed, the centroid method is uniformly used to complete accurate defuzzification calculations, eliminating control output jump problems and outputting two sets of high-precision continuous control quantities for the actuators.

[0069] In this embodiment, the lightweight neural network operation mode is specifically designed for low-end embedded controllers. The network adopts a simplified three-layer fully connected network architecture, with a total of no more than 400 trainable weights. The model's floating-point operation is extremely small, resulting in very low overall computing power overhead. It does not require a large amount of computing resources from the industrial control computer and can stably adapt to the low-computing-power operating environment of embedded low-end controllers, ensuring that even low-end hardware can achieve precise sorting and control. In this embodiment, after the controller completes the working condition analysis, parameter calculation, and generates standardized control instructions, it issues control instructions via high-speed transmission through the TCP industrial bus. The bus transmission latency is extremely low, and it can separately control two types of core actuators: a continuously steplessly adjustable sorting plate with an aperture of 1mm to 10mm and a variable frequency speed-regulating blower. It can adjust the sorting and screening aperture size and the air pressure and speed of the pneumatic conveying pipeline as needed in real time, precisely matching the current material sorting process requirements.

[0070] In this embodiment, the system incorporates a closed-loop feedback correction mechanism. The system continuously collects post-processed data on the particle size and powder content of the sorted finished stone coal in real time. This post-processed data is used as the closed-loop feedback correction amount and input into the adaptive threshold controller. The controller network weights and fuzzy subset membership core parameters are iteratively optimized to dynamically and adaptively correct the overall sorting control strategy. This automatically adapts to changes in random working conditions such as feed particle size fluctuations, carbon content fluctuations, and conveying air pressure disturbances, thus avoiding deterioration of the sorting effect.

[0071] In this embodiment, by switching between fuzzy inference and lightweight neural network control modes, the output control quantity is calculated based on the global fuzzy rules, the bus-linked actuator operates, and the control parameters are iteratively corrected by combining the subsequent material data, thus completing the millisecond-level automated closed-loop sorting of the entire process.

[0072] Example 2 This invention provides a stone and coal particle size detection and sorting control device 10 based on machine vision deep learning, such as... Figure 2 As shown, the device includes: The acquisition unit 100 is used to detect the flow state of the material in the pipeline through an electrostatic trigger sensor, generate a trigger signal to control the synchronization of the high-speed camera and the high-frequency stroboscopic light source, and acquire dynamic particle images in the conveying pipeline. The preprocessing unit 200 is used to perform dust removal and enhancement preprocessing on the acquired image, remove coal dust occlusion and enhance particle edge contours, input the preprocessed image into the lightweight instance segmentation model, perform pixel-level segmentation of particles through adaptive high-resolution branch, and output the mask and bounding box of each particle. The calculation unit 300 is used to calculate the equivalent particle size of each particle based on the segmentation results, perform particle size distribution statistics according to the preset particle size range, and estimate the carbon content of the material by combining the soft measurement model of carbon content. The sorting control unit 400 is used to generate sorting parameter adjustment instructions through an adaptive threshold controller based on the particle size distribution statistics and carbon content estimation, and to perform sorting control through an actuator to obtain real-time corrected controller parameters.

[0073] Regarding the apparatus in the above embodiments, the specific manner in which each unit performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0074] Example 3 To implement the methods of the above embodiments, the present invention also provides an electronic device, which includes a memory and a processor; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the various steps of the methods described above.

[0075] Example 4 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0076] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0077] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0078] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for controlling the detection, sorting, and control of stone and coal particle size based on machine vision deep learning, characterized in that, include: S1 detects the flow state of the material in the pipeline through an electrostatic trigger sensor, generates a trigger signal to control the synchronization of the high-speed camera and the high-frequency stroboscopic light source, and acquires dynamic particle images in the conveying pipeline. S2, perform dust removal and enhancement preprocessing on the acquired image to remove coal dust occlusion and enhance particle edge contours. Input the preprocessed image into the lightweight instance segmentation model, perform pixel-level segmentation of particles through adaptive high-resolution branch, and output the mask and bounding box of each particle. S3, based on the segmentation results, calculate the equivalent particle size of each particle, and perform particle size distribution statistics according to the preset particle size range. At the same time, combine the carbon content soft measurement model to estimate the carbon content of the material. S4, based on the particle size distribution statistics and carbon content estimation, generates sorting parameter adjustment instructions through an adaptive threshold controller, and performs sorting control through an actuator to obtain real-time corrected controller parameters.

2. The method according to claim 1, characterized in that, The process of detecting the flow state of materials within the pipeline using an electrostatic trigger sensor, generating a trigger signal to synchronize a high-speed camera with a high-frequency stroboscopic light source, and acquiring dynamic particle images within the conveying pipeline includes: The non-contact ring electrode electrostatic sensor installed upstream of the observation section is used to detect the charged state of the material in the pipeline in real time. When the material is detected to have reached the sensor detection area, a TTL pulse signal is output to the external trigger input terminal of the camera controller within 10 microseconds. Based on the real-time flow rate of the material and the distance between the sensor and the observation section, the trigger delay compensation amount is calculated through the flow rate-delay correspondence table to generate a timing-calibrated trigger signal. The trigger signal is sent to a high-frequency LED strobe light source and a high-speed industrial camera, and the pulse width of the light source is controlled to synchronize with the camera exposure time so that the camera completes the exposure when the particle is in the optimal imaging position, and acquires dynamic particle images in the conveying pipeline.

3. The method according to claim 1, characterized in that, The acquired image undergoes dust removal and enhancement preprocessing to remove coal dust occlusion and enhance particle edge contours. The preprocessed image is then input into a lightweight instance segmentation model, which performs pixel-level segmentation of the particles through an adaptive high-resolution branch, outputting a mask and bounding box for each particle, including: The dark channel map is calculated for the acquired image. The average value of the original image pixels corresponding to the top 0.1% of the brightest pixels in the dark channel map is taken as the atmospheric light value. The transmittance map is estimated and a soft matting algorithm is used to refine the transmittance map to complete the dust removal image restoration. The dust-removed grayscale image is divided into 8×8 non-overlapping sub-blocks. A histogram is calculated for each sub-block. The contrast limit threshold is set to 4 times the number of gray levels. Histogram portions exceeding the threshold are truncated and redistributed. Histogram equalization is performed on each sub-block. Bilinear interpolation is used to eliminate discontinuities at the sub-block boundaries to obtain the enhanced image. The system identifies and removes blank frames without material, overexposed or underexposed invalid frames, and inputs the valid images into the lightweight instance segmentation model.

4. The method according to claim 1, characterized in that, The lightweight instance segmentation model uses YOLOv8-seg as its backbone network. The YOLOv8-seg network consists of three parts: a backbone network, a neck network, and a segmentation head. The backbone network adopts a CSPDarknet structure, containing 5 convolutional layers and 4 C2f modules. The neck network adopts a PANet structure, which enhances multi-scale object detection capabilities through top-down and bottom-up feature fusion. The segmentation head includes a detection branch and a segmentation branch, which output the target bounding box, category, and pixel-level mask, respectively. An adaptive high-resolution branch is added after the last C2f module of the backbone network. This branch contains one 2x upsampling layer, one 3×3 convolutional layer, and one feature fusion module.

5. The method according to claim 1, characterized in that, The process involves calculating the equivalent particle size for each particle based on the segmentation results, statistically analyzing the particle size distribution within a preset particle size range, and simultaneously estimating the carbon content of the material using a soft-sensing model. This includes: The camera intrinsic and extrinsic parameters and distortion correction were completed using a checkerboard calibration board via binocular vision calibration, and the pixel equivalent was determined. Extract the boundary pixel coordinates of each particle mask, fit the ellipse parameters using the direct least squares method, calculate the major axis a, minor axis b, and rotation angle θ, and calculate the equivalent particle size using the geometric mean. Simultaneous calculation of length-to-diameter ratio As an auxiliary parameter for particle morphology; Histogram statistics are performed in five ranges: 0-1mm, 1-3mm, 3-6mm, 6-10mm, and greater than 10mm. The statistical period is adaptively set to 100ms-1s, and the P10, P50, and P90 particle size values ​​and powder content are calculated simultaneously.

6. The method according to claim 1, characterized in that, The process involves generating sorting parameter adjustment commands through an adaptive threshold controller based on particle size distribution statistics and carbon content estimates, and then controlling the sorting process through an actuator to obtain real-time corrected controller parameters, including: The powder content, P50 particle size value, and carbon content estimate are input into the adaptive threshold controller. The controller is implemented using fuzzy inference. First, the powder content, P50, and carbon content are divided into three fuzzy subsets: low, medium, and high, and a triangular membership function is used. The size of the sorting plate hole is divided into five fuzzy subsets: very small, small, medium, large, and very large, and a trapezoidal membership function is used. 25 complete fuzzy rules are formulated. The centroid method is used to complete the defuzzification and output the precise control values ​​of the sorting plate hole size and the pneumatic conveying speed. The control system transmits data to the adjustable aperture sorting plate and the variable frequency fan actuator via the TCP industrial bus, synchronously collecting particle size data of the sorted material as feedback, and correcting the controller parameters in real time. The entire closed-loop control cycle does not exceed 33ms.

7. The method according to claim 1, characterized in that, Also includes: During the model training phase, a conditional generative adversarial network is used to augment the data of wet particle adhesion and overlap scenarios. The generator takes a single particle image and particle size label as input and outputs a synthetic sample of adhered or overlapping particles. The discriminator distinguishes between real samples and synthetic samples. The generation effect is optimized through adversarial training. During training, real samples and synthetic samples are mixed in a 7:3 ratio to expand the diversity of the dataset.

8. A stone and coal particle size detection and sorting control device based on machine vision deep learning, characterized in that, include: The acquisition unit is used to detect the flow state of the material in the pipeline through an electrostatic trigger sensor, generate a trigger signal to control the synchronization of the high-speed camera and the high-frequency stroboscopic light source, and acquire dynamic particle images in the conveying pipeline. The preprocessing unit is used to perform dust removal and enhancement preprocessing on the acquired image, remove coal dust occlusion and enhance particle edge contours. The preprocessed image is input into the lightweight instance segmentation model, which performs pixel-level segmentation of particles through an adaptive high-resolution branch and outputs the mask and bounding box of each particle. The calculation unit is used to calculate the equivalent particle size of each particle based on the segmentation results, perform particle size distribution statistics according to the preset particle size range, and estimate the carbon content of the material by combining the soft measurement model of carbon content. The sorting control unit is used to generate sorting parameter adjustment instructions through an adaptive threshold controller based on the particle size distribution statistics and carbon content estimation, and to perform sorting control through an actuator to obtain real-time corrected controller parameters.

9. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images